Azure AI Custom Vision: Fix Setup, Training & API Errors

Start at the source of truth: the Azure Portal itself. Browse to portal.azure.com, search for "Custom Vision" in the top search bar, and open your Custom Vision resource. You should see two resources if you set things up correctly, one with a kind of CustomVision.Training and one with CustomVision.Prediction. If you only see one, that's your first problem.

Click your Training resource and navigate to Keys and Endpoint in the left sidebar. Copy the Endpoint URL, it should look like:

https://YOUR-RESOURCE-NAME.cognitiveservices.azure.com/

Now open the Custom Vision web portal at customvision.ai in Microsoft Edge or Google Chrome (these are the only officially supported browsers, running it in Firefox or an older Edge version is an unsupported configuration that causes unpredictable portal behavior). Sign in with the same Microsoft account tied to your Azure subscription.

Click the gear icon at the top right. Under "Accounts," you should see your Azure resources listed. If your subscription doesn't appear here, click Add subscription and enter your Azure directory and subscription details manually.

If everything looks connected but you're still getting errors, scroll down to the "Limited Trial" section. If you're on the free Limited Trial (not tied to an Azure subscription), you're capped at two projects and 5,000 training images total. Migrate to an Azure subscription-backed resource to get past these limits.

When this step is done correctly, the Custom Vision portal should show your projects listed under the correct Azure resource, and hovering over the resource should show its region matching what's in Azure Portal.

Authentication errors are the number-one cause of Azure Custom Vision training failures when using the SDK or REST API directly. The symptom is typically an HTTP 401 Unauthorized or HTTP 403 Forbidden when calling the Training API endpoint.

The Training API requires the Training-Key header, not an Authorization Bearer token, not an API key in the query string. Here's the correct header format for a REST call:

POST https://YOUR-RESOURCE.cognitiveservices.azure.com/customvision/v3.3/training/projects
Training-Key: YOUR-TRAINING-KEY
Content-Type: application/json

A very common mistake is using the Prediction key in the Training-Key header. They look identical (32-character hex strings) but they're resource-specific. Double-check by going to Azure Portal → your Training resource → Keys and Endpoint. The key shown there is your Training key. Then go to your Prediction resource and note that key separately.

If you're using one of the official SDKs (.NET, Python, Java, Go), the client constructor takes your Training endpoint and Training key explicitly:

# Python SDK example
from azure.cognitiveservices.vision.customvision.training import CustomVisionTrainingClient
from msrest.authentication import ApiKeyCredentials

credentials = ApiKeyCredentials(in_headers={"Training-key": TRAINING_KEY})
trainer = CustomVisionTrainingClient(ENDPOINT, credentials)

If you're getting a ProjectNotFoundException or 404 Not Found when trying to access an existing project, the project GUID in your code may be wrong. Get the correct project ID from the Custom Vision portal: open your project, and look at the URL, it contains the project GUID in the format /projects/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

Once authentication is correct, the Training API call should return an HTTP 200 with the project details JSON. If you see a 429, you've hit the rate limit, back off and retry with exponential backoff.

This one takes more finesse than a simple config fix. If your Azure Custom Vision image classification model is training but the accuracy numbers are disappointing, say, below 80% precision and recall on your quick test, the problem usually isn't the service. It's the training data or domain selection.

First, check your domain setting. The Custom Vision service offers several specialized algorithm domains optimized for specific subject material: General, Food, Landmarks, Retail, Adult, and more. If you're classifying retail product images but your project is set to the General domain, you're leaving accuracy on the table. In the Custom Vision portal, go to your project's Settings tab (the gear icon inside the project) and change the Domain. Changing the domain requires retraining from scratch, but it's worth it.

For image count, the official guidance is to start with 50 images per label. I'd say 50 is the floor, not the target. For reliable accuracy in a production context, I push clients toward 100–200 images per label, with genuine variation in lighting, angle, background, and scale. Duplicate or very similar images hurt more than they help, the model overfits to your training set instead of generalizing.

Also watch for class imbalance. If you have 200 images of "cat" and 15 images of "dog," the model will learn to predict "cat" aggressively. Balance your label counts as evenly as you can.

The Custom Vision portal's Smart Labeler feature can accelerate labeling after initial training. Once you have a trained iteration, Smart Labeler suggests tags for new untagged images. Go to the Training Images tab, select untagged images, and click the Smart Labeler button. Review its suggestions, don't blindly accept them, then use those accepted labels to retrain.

One hard limit from official documentation: Azure Custom Vision is not designed for detecting subtle differences in images, such as minor cracks or dents in quality assurance scenarios. If that's your use case, you'll get frustratingly inconsistent results regardless of how much data you add. Consider Azure Machine Learning AutoML for those scenarios instead.

You've trained your model, you've published your iteration, you're making prediction calls, and you get either a 404, a blank response, or confidence scores that are all near zero. Here's how to work through this.

First, confirm the prediction URL format. The Prediction API endpoint has a specific structure that trips people up constantly:

POST https://YOUR-PREDICTION-RESOURCE.cognitiveservices.azure.com/customvision/v3.0/Prediction/{projectId}/classify/iterations/{publishedName}/image

There are two variants, one for sending an image file (binary) and one for sending an image URL. Make sure you're hitting the right one. For a URL-based call, append /url and send JSON with the Url field. For a binary image, send the raw bytes with Content-Type: application/octet-stream.

The {publishedName} segment must exactly match the name you gave the iteration when you published it, it's case-sensitive. Go to the Performance tab in the Custom Vision portal, find your iteration, and check the "Published as" name. Copy it character for character.

For authentication on the Prediction API, use the Prediction-Key header (not Training-Key):

Prediction-Key: YOUR-PREDICTION-KEY
Content-Type: application/octet-stream

If predictions come back with all confidences near 0.0 or 0.33 (for a 3-class model), this suggests the model never truly trained on meaningful differences. Check that your training images are genuinely distinct across labels and that you have enough of them. Also verify you're calling the right project ID, calling a classification model's endpoint with object-detection-formatted expectations (or vice versa) produces garbage output, not an error.

A successful prediction response looks like this:

{
  "predictions": [
    {
      "tagName": "Cat",
      "probability": 0.9973,
      "tagId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    }
  ]
}

If you see this JSON structure with sensible probability values, the Prediction API is working correctly.

Exporting a trained Azure Custom Vision model for offline inference is a common need, whether you're deploying to a mobile app, an edge device, or a Windows ML application. This step only works if you trained your project with an exportable (compact) domain. The standard "General" domain is not exportable. You must use "General (compact)" or another compact variant.

If you trained on a non-compact domain and now want to export, you'll need to create a new project with the compact domain, re-upload your images, and retrain. There's no way to convert an existing non-compact model to an exportable format after the fact.

Assuming you used a compact domain, here's how to export from the portal:

1. Go to your project in the Custom Vision portal
2. Click the Performance tab
3. Select the trained iteration you want to export
4. Click Export
5. Choose your target format: CoreML (iOS), TensorFlow (Android/Python), ONNX (Windows ML), Docker, or Vision AI DevKit

For programmatic export via the Training API:

# Python SDK, export as ONNX
export = trainer.export_iteration(project_id, iteration_id, "ONNX")
while export.status == "Exporting":
    time.sleep(1)
    export = trainer.get_exports(project_id, iteration_id)[0]
    
# Download the exported model
response = requests.get(export.download_uri)
with open("model.onnx", "wb") as f:
    f.write(response.content)

For Windows ML integration, take the downloaded .onnx file and add it to your Visual Studio project as a content file. Windows ML generates a wrapper class automatically when you add an ONNX file to a UWP project. The generated class handles input tensor preparation and output parsing, you just call EvaluateAsync() with your image data.

For TensorFlow export targeting Python inference, load the SavedModel directory and run predictions using the standard tf.saved_model.load() approach. The official docs include a Python sample for exactly this use case, it's worth pulling from the Azure Samples GitHub repository rather than writing the tensor reshaping code from scratch.