Materials in Paintings Dataset is a large scale annotated dataset containing 19k paintings with 200k+ bounding boxes. Objects and materials in the paintings were annotated. My colleagues collected data and built the dataset. We use art, especially oil paintings, to study human perception.

▼ Example painting annotated with multiple bounding boxes indicating different materials

Challenges

• Paintings have diverse aspect ratios, the layout should adapt to display them properly.
• Unlike traditional datasets, this novel dataset contains not only images of paintings but also annotated bounding boxes in most images. These annotations highlight various materials—such as fabric from clothing or wood from furniture—and their spatial relationships within the original painting. The interface must be able to display the annotations.
• Users should be able to search for paintings using various properties, including contained materials, painting title, artist, and creation year (or a year range).
• The web interface should accommodate both professional and casual users. Professional researchers may want to fine-tune search criteria or download the entire dataset for research or training purposes; while casual users are more likely to engage in exploratory browsing.

Solution

Key features:

• Masonry layout to display artworks with various aspect ratios
• Fixed search box for general search
• Expandable advanced search area for advanced search, but not taking screen space when not in use
• Can search both whole paintings and bounding boxes
• Filters for materials
• Expandable options marked with underlines
• Search by creation year
• Slider for quick selection, number input for precise search
• Detail page showing title, artist, bounding boxes, link to the original museum repository and download button
• Separate download function for easy and quick access for professional users