How to Classify Images by Using Microsoft Cognitive Services - Custom Vision Service

What is Custom Vision Service?

Custom Vision Service is a tool for building custom image classifiers and making them better over time. It makes it easy and fast to build, deploy, and improve an image classifier. Images can be uploaded and trained via a web interface and a REST API. The service can be found under

What is it good for / what not?

Custom Vision Service works best when the items that should be classified are prominent in the images. It does image classification but not yet object detection.

Very few images are required to create a classifier. Around 30 images per class is enough to start with a prototype.

It is not useful in scenarios where you must detect very small differences in images (minor cracks in quality assurance scenarios).

Getting Started

In our sample, we want to build a simple model, classifying playing cards. Therefor we will train our model with 10 images for clubs, diamonds, hearts and spades. These are the necessary steps:

  1. Go to and login with a valid Microsoft account. You will be asked now to create new project. If you create a project you are asked to provide a name, a description and to pick a domain your model belongs to.


  1. Add images to your classifier. This can be done in the „Training Images“ area. Click on „Add Images“. After that you can choose some local files. Those can be tagged before they will be uploaded. In the web app you can only upload files from your local computer. It is also possible to load training images using the REST API. There it is possible to use images from URLs


  1. Train Classifier.

After uploading the images, you can train the classifier. All you must do is select the Train button.

Each time you select the Train button, you create a new iteration of your classifier. You can view all your old iterations in the Performance tab, and you can delete any that might be obsolete. When you delete an iteration, you end up deleting any images that are uniquely associated with it.

  1. Evaluate your Classifier

When your model is trained you find information needed to evaluate your classifier under Performance

For each iteration, the precision and recall indicators are telling you how good your classifier is. These indicators are built by using the so called k-fold cross validation.


You have built your first image classifier!


Test the model

To test your image classifier, use the Quick Test button. There you can chose an image URL or a local file. The model will be used and results and probabilities will be shown.


Retrain the model

Under the tab Predictions you’ll find the images used during the quick test. You can select and relabel them. To retrain the model, click on Train


The Custom Vision service automatically creates a REST API for you project. In the tab Performance click on Prediction URL to get the needed URLs and prediction keys.

The REST API can be used with image URLs and by sending a file.


  • Erstellt am .
Copyright by Orange Networks GmbH