Thermal imaging is an imaging method that uses infrared radiation. This radiation is emitted by all objects with a temperature above absolute zero. One of the many uses of thermal imaging may be material classification. For example, where images can be classified according to the apparent material in the picture. Material classification can be very beneficial in many fields, such as recycling, robotics, material sorting, security industry, etc. Some of the advantages of thermal classification are the ability to work in a wide range of lighting conditions, and the existence of information not available in standard RGB images. Over recent years, the field of thermal imaging has greatly developed, and this technology has now become more accessible and affordable. For these reasons, the SIPL lab purchased the FLIR-ONE-PRO camera. FLIR-ONE-PRO is a small and mobile thermal camera that can be connected to a mobile phone. Using this camera, one can take thermal images, RGB images, and fusion images (RGB and thermal images combined). Our goal was to classify thermal images of materials using deep learning methods. We classified materials with different CNN architectures, using different kinds of images. We experimented with different methods, such as transfer learning, image processing and CNN visualization. In addition, we compared the results we obtained from the different architectures and the different kinds of images. Our focus was to compare the classification of thermal data with the classification of RGB data.