Other people in our office spent their youth playing Magic The Gathering (with beautiful artworks), or collecting sports cards with their favorite athletes. In my country, basketball cards and ice hockey cards were really popular. Cards are still loved, played, collected, and traded by geeks, collectors, and sports fans across the world! Their market is growing, and so is the need for automation of image processing on websites and apps for collectors. Right now, cards can be seen even as a great investment.

Where can you use visual AI for cards?

Trading card games (トレーディングカード) can consist of tens of thousands of cards. In principle, building a basic image classifier based solely on image recognition leads to low precision and is simply not enough for more complicated problems.

However, we are able to build a complex similarity system that can recognize, categorize, and find similar cards by a picture. Once trained properly, it can deal with enormous databases of images it never encountered before. With this system, you can find all the information, such as the year of release, card title, exact value, card set, or whether it already is in someone’s collection, with just a smartphone image of the card.

Tip: Check out our Computer Vision Platform to learn about how basic image recognition systems work. If you are not sure how to develop your card search system, just contact us and we will help you.

Collectibles are a big business and some cards are really expensive nowadays. Who knows, maybe you have the card of Charizard or Kobe Bryant hidden in your old box in the attic. We can develop a system for you that can automatically analyze the bulk of trading cards sent from your customers or integrate it into your mobile/smartphone app.

Automatic Recognition of Collectibles

Ximilar built an AI system for the detection, recognition and grading of collectibles. Check it out!

Read more

What can visual search do for the trading cards market?

In the last year, we have been building a universal system able to train visual models with numerous applications in image search engines. We already offer visual search services for photo search. But, they are optimized mostly for general and fashion images. This system can be tuned to trading cards, coins, furniture & home decor, arts, and real estate, … there are infinite use cases.

In the last decades, we have all witnessed the growth of the TCG community. However, technologies based on artificial intelligence have not yet been used in this market. Plus, even though the first system for scanning trading cards was released by ebay.com, it was not made available for small shops as an API. And since trading card games and visual AI are a perfect match, we are going to change it – with a card image search.

Tip: Check out Visual Product Search to learn more about visual search applications.

Which TCG cards could visual AI help with?

An image search engine is a great approach when the number of classes for the image classification is high (above 1,000+). With TCGs, each card represents a unique class. A convolutional neural network (CNN) trained as a classifier can have poor results when working with a larger number of classes.

Pokémon TCG contains more than 10,000 cards (classes), Magic the Gathering (MTG) over 50.000, and the same goes for basketball or any other sports cards. So basically, we can build a visual search system for both:

• Trading card games (Magic the Gathering, Lord of the Rings, Pokémon, Yu-Gi-Oh!, One Piece, Warhammer, and so on)
• Collectible sports cards (like Ice Hockey, Football, Soccer, Baseball, Basketball, UFC, and more)

A visual search/recognition technology is starting to be used on E-bay when listing trading and sports cards for sale. However, this is only available in the e-bay app on smartphones. The app has a built-in scanning tool for cards and can find the average price with additional info.

Our service for card image search can be integrated into your website or application. And you can simply connect via API through a smartphone, computer, or sorting machine to find exact cards by photo, saving a lot of time and improving the user experience!

Our Pokémon Card Image Search

We’ve been recently training an AI (neural network) model for Pokémon trading cards, Yugioh! and Magic The Gathering. Why these? Pokémon is the most played TCG in the world, the game has simple rules and an enormous fan base. Very popular are also MTG and Yugioh! Some cards are really expensive, but more importantly, they are traded heavily!

With this model, we built a reverse search for finding the exact Pokémon card, MTG and Yugioh! cards, which achieved 94%+ accuracy (i.e. exact image match). And we are still talking about a prototype in beta version that can be improved to almost 100 %. This search system can return you the edition of the card, language, name of the card, year of release and much more.

If you would like to try the system on these three trading card games, then the endpoint for card identification (/v2/tcg_id) from the Collectibles Recognition service is the right choice for you. If you need to tune it on your image collections or have any other games or cards (sports) then just contact us and we can build a similar service for you.

Automatic grading and inspection of cards with AI

A lot of companies are grading sports & trading cards manually. Our visual AI can be trained to detect corner types, scratches, surface wear, light staining, creases, focus, and borders. The Image recognition models are able to identify marks, wrong cut, lopsided centering, print defects and other special attributes.

For example, PSA is a company that has developed its own grading standards for automatic card grading (MINT). With our platform and team, you can automatize the entire workflow of grading with just one photo. We provide several solutions for computing card grades and card condition.

Customized solution for visual search

With the new custom similarity service, we are able to create a custom solution for trading card image search in a matter of weeks. The process for developing it is quite simple:

1. We will schedule a call and talk about your goals. We will agree on how we will obtain the training data that are necessary to train your custom machine-learning model for the search engine.
2. Our machine-learning specialists will assemble a testable image search collection and train a custom machine-learning model for you in a matter of weeks.
3. After meeting all the requirements of PoC, we will deploy the system to production, and you can connect to it via Rest API.

Image Recognition of Collectibles

Machine learning models bring endless possibilities not only to pop culture geeks and collectors, but to all fields and industries. From personalized recommendations in custom fashion search engines to automatic detection of slight differences in surface materials, the visual AI gets better and smarter every day, making routine tasks a matter of milliseconds. That is one of the reasons why it is an unlimited resource of curiosity, challenges, and joy for us, including being geeks – professionally :).

Ximilar is currently releasing on a ready-to-use computer vision service able to recognize collectibles such as TCG cards, coins, banknotes or post stamps, detect their features and categorize them. Let us know if you’d like to implement it on your website!

If you are interested in a customized AI solution for collector items write us an email and we will get back to you as soon as possible. If you would like to identify cards with our Collectibles Recognition service just sign up via app.ximilar.com.

The post Pokémon TCG Search Engine: Use AI to Catch Them All appeared first on Ximilar: Visual AI for Business.

The technology behind image similarity is robust, reliable & fast. Built on state-of-the-art (SOTA) AI models and vector databases, you can search millions of images/products in milliseconds. It is used by big e-commerce players as well as small startups for showing visual alternatives or finding products with pictures. Some of our customers have hundreds of millions of images in their collections and do more than 100 million requests per month. Let’s dive into building a superfast similarity search service for your web.

What is Image Similarity?

Image Similarity, or image similarity search, is a visual AI service comparing, grouping, and recommending visually similar images. For example, a typical use case is a product recommendation of similar items in e-shops. It can also be used for reverse image search, where the query is an external image and the results are images from the collection. This approach gives way more accurate results than searching by tags, labels and other attributes.

Ximilar is using state-of-the-art deep learning models for all visual search services. We build our own indexing & searching technology that can run both as a service or on your hardware if needed. The collections can be focused either on product photos, fashion, image matching, or generic photos (stock images).

Features of the Image Similarity Service

Here are several features of the Image Similarity service that we think are crucial:

• Simple access through the Ximilar App (creating a collection on click) and connection to REST API
• The scalable search service can handle collections with hundreds of millions of similar items (images, videos, etc.) and hundreds of requests per second with both CRUD operations and searching
• The ultra-fast and reliable engine that is mostly deployed in large e-commerce platforms – the query for finding the most visually similar product is low latency (in milliseconds)
• The service is customizable – the platform enables you to train your own model for visual similarity search
• Advanced filtering that supports JSON meta-data – if you need to restrict the result to a specific field
• Grouping based on similarity – our search technology can group photos of the same product as one item
• Security and privacy of your data – only meta-data and the visual representation of the images are stored, therefore your images are not stored anywhere
• The service is affordable and cost-effective both for startups and enterprises, offering free plan for tests as well as discounts with your growth over time
• We can deploy it on your hardware, independently of our infrastructure, and also offline – custom similarity model and deployment appropriate to your needs
• Our search engine and machine learning models improve constantly – maintaining much higher quality than any other open-source project & we are able to build custom search engines with trained models

Applications Using Visual Similarity

According to this research by Deloitte, merchandising with artificial intelligence is more and more relevant, and recommendation engines play a vital part in it. Here are a few use cases for visual similarity engines:

• E-shops that use product similarity to help customers to browse and find related products (e.g. in fashion & luxury items, home decor & furniture, art, wall art, prints & posters, collectible trading cards, comics, trademarks, etc.)
• Stock photo databases suggesting similar content – getting visual alternatives of photos, designs, product images, and videos
• Finding the exact products – apps like Vivino for finding wine or any kind of product are easy to develop for us
• Visual similarity duplicate finder (also image matching or deduplication), to know which images are already in your database, or which product photos you can merge together
• Reverse image search – finding a product or an image with a picture online
• Finding similar real estate based for example on interior design, furniture, garden, etc.
• Comparing two images for similarity – for example patterns or designs

Recommending products to your customers has several advantages. Firstly, it creates a better user experience and helps your customers find the right products faster. Secondly, it instantly makes the purchase rate on your web higher. This means a win on both sides – satisfied customers and higher revenue for you. Read more about customer experience and product recommendations in our blog post on fashion search.

Step by step: Building Real-Estate Image Search

Creating the Collection

So let’s take a look at how to easily build your own similarity search engine with the Ximilar platform. The first step is to log in to the Ximilar App. If you don’t have an account, then sign up – it’s free and takes just a minute. After that, on the Dashboard, click on the Visual Search tile and then the Image Similarity service. Then go to the Collections in the left menu and click on Create New Collection. It will show a pop-up with different collection types from which you need to select one.

The collection is a space where you upload your images. With this collection, you are performing queries for search. You can choose from Generic Photo Collection, Product Photo Collection, Dominant Colors Similarity, and Image Matching. Clicking on one of the cards will create a collection for your account.

Each of these collection types is suitable for different types of images:

• Use Generic Photos if you work with stock photos
• Pick a Product Photos collection if you are an e-commerce company
• Select Image Matching to find duplicates in your images
• For the fashion sector, we recommend using a specialized service called Fashion Search
• Custom Similarity is suitable if you are working with another type of data (e.g. videos or 3D models). To do this, please schedule a call with us, and we will develop your own model tuned for your data. For instance, we built a photo search system for the Magic the Gathering Trading Cards for one of our customers.

For this example of real estate, I will use a Generic Photo Collection. The advantage of Generic Photo Collection is that it also supports searching images via text input/query. We usually develop custom similarity models for real estate, when the customers need specific and more accurate results. However, for this simple use case, the generic real estate model will be enough.

Format of Image Similarity Dataset

First, we need to prepare a text file with JSON records. Each record represents an image that we want to store/insert into our collection. The key field is "_url" with the image URL. The advantage of the _url is that you can directly see and inspect the results via app.ximilar.com.

You can also optionally send records with base64 data, this is great if your data are stored locally on your computer. Don’t worry, we are not storing the whole images (data or base64) in the collection database, just URLs with all other metadata present in the records.

The JSON records look like this:

{"_id": "1_1", "_url": "_URL_IMAGE_PATH_", "estate_id": "1", "category": "indoor", "subcategory": "kitchen", "tags": []}
{"_id": "1_2", "_url": "_URL_IMAGE_PATH_", "estate_id": "1", "category": "indoor", "subcategory": "kitchen", "tags": []}
...

If you don’t have image URLs, you can use either "_file" or "_base64" fields for the image data (locally stored "_file" data are automatically converted by the Python client to base64). The image similarity engine is indexing every record of the collection by extracting a representation from the image by a neural network model. However, we are not storing the images in our engine. So, only records that contain "_url" will be visualized in the Ximilar App.

You must store unique identifiers of each image in the "_id" field to identify your images in the collection. The value of this field must be a string. The API endpoint for searching is returning this _id values, that is how you get the results for visual search. You can also store additional fields for every JSON record, and then you can use these fields for filtering, grouping, and tuning the similarity function (see below).

Filling the Collection With Your Data

The next step requires a few lines of code. We are going to insert the prepared images into our collection using our python-client library. You can install the library using pip or directly from GitLab. The usage of the client is very straightforward and basically, you can just use the script tools/collections/insert_json_records.py:

python insert_json_records.py --type generic --auth_token __YOUR_TOKEN__ --collection_id __COLLECTION_ID__ --path /path/to/the/file.json

You will find the collection ID and the Authorization token on the “collection page” in the Ximilar App. This script will run for a few minutes, depending on the size of your image dataset.

Result: Finding Visually Similar Pictures

That was pretty easy, right? Now, if you go to the collections page, you will see something like this:

All images from the JSON file were indexed, and now you can inspect the collection in the Ximilar App. Select the Similarity Search in the left menu of the Image Similarity service and test how the similarity works. You can specify the query image either by upload, by URL, or your IDs, or by choosing one of the randomly selected images from the collection.

Even though we have indexed just several hundred images, you can see that the similarity engine works pretty well. The first image is the query image and the next images are the k-nearest to the query image:

Rest API Connection for Image Search

The next step might be to integrate the service into your application via API. You can either directly use the REST API for searching visually similar images or, if you are using Python, we recommend our Python SDK client like this:

# pip install ximilar-client
from ximilar.client import SimilarityPhotosClient
client = SimilarityPhotosClient("_API_TOKEN_", "_COLLECTION_ID_")
# search k nearest items
client.search({"_id": "1"}, k = 3)
# search by external image
client.search({"_url": "_URL_PATH_"})

Advanced Features for Photo Similarity

The search for visually similar images can be combined with filtering on metadata. This metadata can be stored in the JSON, as in our example with the "category" and "subcategory" fields. In the API, the filtering is specified using a MongoDB-like syntax – see the documentation.

For example, let’s say that we want to search for images similar to the image with ID=1_1 that are indoor photos made in a kitchen. We assume that this meta-information is stored in the “category” and “subcategory” fields of every JSON record. The query will look like this:

client.search({"_id": "1_1"}, filter={"category": "Indoor", "subcategory": "Kitchen"})

If we know that we will often filter on some fields, we can specify them in the “Fields to index” option of the collection to make the query processing more efficient.

Often, your data contains several photos of one “object” – a product or, in our example, real estate. Our service can group the search results not by individual photos but by product IDs. You can set this in the advanced options of the collection by specifying the name of the real estate in the Product ID field, and the magic will happen.

Enhancing Image Similarity Engine with Tags

The image similarity is based purely on the visual content of the image. However, you can use your tags (labels, keywords) to enhance the similarity search. In the example, we assume that the data already contains categories, subcategories, and tags. In order to enhance the visual similarity search with tags, you can fill the “tags” field for every record with your tags, and also use method /v2/visualTagsKNN. After that, your search results will be based on a combination of visual similarity and keywords.

If you don’t have categories and tags, you can create your own photo tagger through our Image Recognition service, and enrich your image data automatically before indexing. The possibilities of image recognition models and their combinations are endless, resulting in highly customizable solutions. Read our guide on how to build your own Image Recognition API.

You can build several models:

• One classifier for categorizing indoor/outdoor/floor plan photos
• One classifier for getting room type (Bedroom, Kitchen, Living room, etc.)
• One tagger for outdoor tags like (Pool, Garden, Garage, House view, etc.)

To Sum Up

The real estate photo similarity search is only one use case of visual similarity from many (fashion, e-commerce, art, stock photos, healthcare…). We hope that you will enjoy working with this service, and we are looking forward to seeing your projects based on it. Thanks to our developers Libor and Ludovit, you can use this service through the frontend app.

Visual Similarity service by Ximilar is unique in terms of search quality, speed performance, and all the possibilities of the API. Our engineers are constantly upgrading the quality of the search, so you don’t have to. We are able to build custom solutions suitable for your data. With multiple collections, you can even A/B test the performance on your websites. This can run in our cloud as SaaS or in your warehouse! If you have more questions about pricing, and technical details,  or you would like to run the similarity search engine on your own machines, then contact us.

The post Image Similarity as a Service For Your Web appeared first on Ximilar: Visual AI for Business.