Image t-SNE live

Code here

This app lets you view a t-SNE of images, as well as its corresponding grid assignment. It is a superset of all the functionality of ImageTSNEViewer, inheriting the basic viewer and export functionality, while also allowing for the actual analysis and t-SNE assignment to be done within the app itself. It is entirely compatible with the export in ImageTSNEViewer and can also load a JSON file generated by ImageTSNEViewer’s analysis script (in Python), or a JSON file generated by the associated guide, image-tsne.

gridded t-SNE of images of animals from CalTech-101

To build from source, the following addons are required:

Analyze new directory

Clicking “analyze directory” will prompt you to select a folder of images to search through recursively to generate a new t-SNE and grid assignment. It will limit the analysis to the number of images specified by the max num images slider, taking a random subset of that size.

Or load a pre-existing t-SNE

Instead you can have it load a JSON file created from the script tSNE-images.py in the scripts folder (follow the instructions in the corresponding guide) or from the associated notebook in ml4a-guides. You can do this by clicking “load result from json”.

Viewer application

Once an image set has been loaded or analyzed from scratch, the viewer displays the images according to the layout. It will look something like the below figure. There are a few parameters for viewing it on the GUI.

  • view as grid toggles whether to view the t-SNE in the grid layout or original t-SNE embedding.
  • scale controls the scaling factor of the point assignments to stretch it out or fit it to screen.
  • image size is a multiplier on the dimensions of the image (it is set automatically)

There are also several parameters which control the analysis.

  • max num images is the maximum limit to how many images to include in the t-SNE analysis.
  • perplexity sets t-SNE’s perplexity factor (roughly the number of clusters to distribute points around)
  • theta controls the accuracy vs speed tradeoff (lower is more accurate)
  • analyze new directory will let you choose a folder of images to do a new analysis on.
  • load result from json will load a JSON file with the results of a previous analysis.
  • save result to json will export the current layout to a new JSON file.
  • save screenshot will save a PNG of the current view of the t-SNE. image-tsne-live-gui

Export JSON

By clicking “save result to json” the app will save a JSON file of the t-SNE coordinates and the corresponding image file paths. You can load it back when you reopen the application in the same way by clicking “load result from json”.

To keep it compatible with the above-mentioned applications and scripts, this export does not save the grid positions, only the original t-SNE, and it recalculates the grid position upon being loaded into the app.

Export screenshot

By clicking “save screenshot” the app will generate a PNG image of the selected view, and save it to a path of your choosing from the save window prompt.