NOVEL CNN APPROACH (YOLO V5) TO DETECT PLANT DISEASES AND ESTIMATION OF NUTRITIONAL FACTS FOR RAW AND COOKED FOODS

Abstract

NOVEL CNN APPROACH (YOLO V5) TO DETECT PLANT DISEASES AND ESTIMATION OF NUTRITIONAL FACTS FOR RAW AND COOKED FOODS ABSTRACT OF THE INVENTION In the recent era- very frequently people come across health issues due to consumption of poor-quality food items– which leads to issues such as food poisoning, vomiting, diarrhea, etc., For a full development of fruits and vegetables, all the nutrients are necessary during its growth. But due to circumstances like soil defects, infections, water scarcity, waterlogging, etc., the vegetables & fruits gets infected with some diseases. So there arises a necessity of a system which inspects for any presence of disease in fruits & vegetables, with reduced manual intervention. Also, in the current generation where consumers are keen to know their calorie intake – as it relates to their diet plan & disease mitigation. Also, more people are consuming fast-foods in frequent intervals. So, it is important for the consumer to understand the calorie & nutritional content of the food at ease, is bit challenging but it would be beneficial. In this project, a system is designed on Python programming language which is an open-source software and the best version of artificial intelligence deep learning model Called Convolution Neural Network (CNN) is used. CNN is used to understand and identify different Fruits & vegetables, detect the presence & type of the disease that infected the fruit/vegetable. CNN is used as both Supervised and unsupervised model in the entire workflow. Also, the system is enhanced to classify fast foods & cooked food, where it estimates the calorie content and macro nutrition details of the food. To identify the details accurately in the system, convolutional neural networks are used and the results are displayed using computer vision techniques. Anaconda navigator software (Jupyter notebook, IDLE) is used in this project. For customized object identification, mainly for real-time objects – a new technique called as YOLO algorithm is used in this project. YOLOv5 will identify the identical features and unique patterns in the given input image. Also, it helps to customize datasets by inclusion of regional foods and snacks data. YOLO v5 technique projects the image with clear rectangular bounding boxes. The entire system of the proposed model provides an accurate, efficient approach using deep learning computer vision techniques in identification of plant diseases and estimation of calories & macro nutrients facts in fruits, vegetables, raw and cooked foods. Cost-effective software & languages are taken into consideration, as this system is intended to be beneficial to farmers, small scale food processing industries and common people.

Specification

Description:
COMPLETE SPECIFICATION

TITLE OF THE INVENTION:
NOVEL CNN APPROACH (YOLO V5) TO DETECT PLANT DISEASES AND ESTIMATION OF NUTRITIONAL FACTS FOR RAW AND COOKED FOODS


FIELD OF THE INVENTION

The field of this invention relates to artificial intelligence, data science and more particularly in the development of image processing technique using convolution neural network (CNN) and YOLO.
BACKGROUND OF THE INVENTION

This invention relates to the development of a novel approach using Convolution neural network (CNN) and integration with YOLO v5 techniques to detect the presence of disease in plants, fruits, vegetables and estimation of nutritional facts for raw and cooked foods.
Spread of diseases in fruits and vegetables are important to the global food security, but their rapid identification remains difficult. Also, in the current generation where people are keen to know about their food, its calorie and its nutrition details for healthy life. This invention covers a novel approach designed to identify and understand different fruits & vegetables, detect the presence of disease, type of the disease that infected the fruit / vegetable. This system is enhanced to classify raw foods, cooked foods, and fast foods - which also estimate the calorie content and nutrition details of the food. For more accurate information, Convolution neural network (CNN) is being used and the results are displayed using Streamlit framework. For any customized object identification, mainly to detect and take consideration of external irrelevant objects in the image (such as human hand, utensils etc.,) and to count the number of objects - a new technique called as YOLO v5 is being integrated in the system - which also makes part of this invention. This invented system doesn't get significant influence due to external factors such as lighting conditions, clustered objects, distance of the image etc., and also this model is designed with consideration of optimal cost requirements with no requirement of super high speed computers, premium costly software.
Hence, a simple, efficient and a cost-effective solution is developed in this invention.

PRIOR ART STATEMENT

1. The Australian Patent No.: AU 2020100953 A4, Date of Patent: July 16th 2020 is about the invention of "Automated food freshness detection using feature deep learning". This invention proposes the novel methodology called deep learning for automatic detection of fruits and vegetable freshness. The dataset consists of fruit, vegetable and images that are captured by digital camera. Image resizing and enhancement is being performed during preprocessing. A Deep learning based Convolutional neural network model called Inception v4 is trained on the images, collected from a digital camera, preprocessed the obtained image, and labelled. To detect the target object, from the background, YOLO v3 network is proposed and classify the fruits and vegetables by building a deep learning-based model called Inception-ResNet-v2. SoftMax uses binary classes as freshness and rotten fruits and vegetables.

However, this system deals with only the classification of the given fruit / vegetable is rotten or fresh stage.

2. The Chinese Patent No.: CN 110969090 A, Date of Patent: July 04th 2020 is about the invention of "Fruit quality identification method and device based on deep neural network". The embodiment of this invention relates to the technical field of image processing, discloses a fruit quality identification method and device based on a deep neural network. The method begins with training according to fruit sample images in a "training sample set" stored in a fruit quality database and sample quality labels corresponding to the fruit sample images to obtain a fruit quality identification result, followed by receiving a fruit image which needs to be analysed. The received image is stored in the storage module which is analysed with the help of trained image set to obtain the fruit quality identification result.

This system however limits itself only with the fruit quality assessment alone, and doesn't cover to analyse its nutritional content.


3. The WIPO (PCT) Patent No.: WO 2018/040105 A1, Date of Patent: March 08th 2018 is about the invention of "System and method for food recognition, food model training method, refrigerator and server". This invention discloses a food recognition system. The system comprises of various components - a refrigerator, a server, wherein the refrigerator comprises an image collection apparatus used for collecting images of food inside the refrigerator. The server acquires the images of food in the refrigerator and recognizes the images according to a food model so as to acquire food information. The food model is a neural network acquired by training by means of a deep learning algorithm. This system recognizes the food inside the refrigerator by using deep learning technology, but not about the quality of the food (whether any of the fruits / vegetables are infected with diseases) or about the calorie & nutritional facts of the food.

4. The Chinese Patent No.: CN 112240842 A, Date of Patent: January 19th 2021 is about the invention of "A domestic food detects sampling device for fruit maturity". This invention relates to a domestic food detection sampling device to understand the fruit maturity. The designed system consists of components such as a central processor, an image capturing device and a detection device (for further extended analysis of sugar content). The control processor is provided with a historical database, and this database stores the categorical information of the fruits. The control processor retrieves the information corresponding to the fruit image from the database according to the fruit image acquired by the image capturing device. The historical database also contains information of the visual appearance and the maturity of the fruit. Then the control processor estimates the maturity of the fruit in accordance to the information available in the historical database. This invention is further extended with a detection device which comprises of a detection probe (used to peel the fruit) and a sugar analyzer.

Thus, this invention is involved with identification of maturity of a fruit using image processing technique and also by using an external hardware detection device it analyses the sugar content of the fruit.
However, the present invention detects the presence & type of diseases in various fruits and vegetables, and also estimates the calorie and nutritional facts of the raw/cooked and fast foods.

From the patent literature survey, it is identified that no prior inventions are about efficient, simple and with affordable cost CNN approach to detect & identify the disease in fruits and vegetables, also estimates the calorie and nutritional facts of the raw/cooked and fast foods without any limitation with handling single / clustered fruit images. Also the system integrates with YOLO v5, to detect and consider the external irrelevant objects in the given image like Spoon, human hand etc.,
OBJECTIVE OF THE INVENTION

1. To develop an effective disease identification solution, with no limited dependence of manual intervention and prior knowledge & experience in disease identification & recognition.

2. To develop a computer-based solution to identify and recognize a fruit / vegetable and also to identify the presence and name of disease in plants / fruit / vegetable.

3. To develop a cost-efficient system which runs in normal computer (no necessity of advanced super computers) and open-source software, which favors for the common rural people & industry.

4. To develop a unified system to estimate the calorie and nutritional facts, and show its elaborate details like carbohydrates, proteins, fats.

5. To enhance the usage experience, identification of irrelevant objects in the given image like spoon, hand etc., - integration of YOLO v5 in the system.

6. To enhance the user-friendliness of the system, by showing the results in a detailed way using streamlit framework.













SUMMARY OF THE INVENTION

The problems mentioned in the background can be solved by developing an efficient and a cost-effective computer vision-based technique, which identifies the type of disease in plants, fruits / vegetables, and also estimates the calorie and nutritional details of raw / cooked and fast foods.
This invention is about the development of a computer vision-based system which utilizes the technological advancements in the field of deep learning (which is a sub-set of machine learning), artificial intelligence and digital image processing. An advanced computer vision technique called as Convolution Neural Network (CNN) is being deployed to detect and identify various fruits, vegetables and to identify & recognize any infection of disease and name of the same. This was one of the main purposes of the system developed with benefits envisaged to farmers & food processing industries to identify & recognize the infected disease in the plant at the first instance itself and enable to take caution & preventive action before mass spread of the disease. Major part of CNN works as supervised learning methods where it collects data from labelled dataset. But the image clustering process is done by CNN which is a unsupervised model. So, both supervised and unsupervised techniques are being used in this developed system.
By utilizing the advantage of the CNN technique, the developed system is further extended from detecting various fruits & vegetables to also to classify on categories (viz., Fruit, Vegetable, Fast Food, Cooked food) and subsequently assess their calorie for a measure of 100 grams along with their sub macro nutrients like carbohydrates, proteins, fiber and fats. The calorie & nutritional facts estimation is done by using 4-4-9 rule. Being a non-destructive approach based on artificial intelligence, this system is least influenced by different environmental light conditions, shape & size of the fruit / vegetable.
To train the model in an efficient way to achieve the best result, the number of epochs were increased and the results are obtained with more accuracy. This effectiveness of the system is tested with similar images which contain look-alike in shape and colour.
For identification of the important features and to count the number of objects present in the image, a deep learning technique called YOLOv5 technique is integrated with the CNN, for enhancement of the system. YOLOv5 provides the results by using rectangular bounding boxes so that the results are stipulated with more clear evidence.
Inorder to get a user-friendly experience, a technology from Streamlit framework is also integrated to the designed system. The input for the Streamlit framework comes from the .pb5 and .h5 stored data of CNN.
DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 represents the overall diagram of the Novel CNN approach (with integration of YOLOv5 and Streamlit framework) to detect & identify the plant diseases and estimate the calorie and nutritional facts of fruits, vegetables, raw & cooked foods.
As one of the prime objectives is to have cost-efficient system, the base CNN architecture was developed in open-source software with support from high end open-source language. The beginning stage with the developed system is training the CNN with the help of datasets collected from various reliable sources. The CNN undergoes training with the datasets collected from multiple sources and by having multiple epochs - the CNN gets well trained. The datasets collected were split into "Train dataset" and "Test dataset", whereas Train dataset is used to train the CNN architecture, and Test dataset is used to review the CNN. The next stage in the system is the Image segmentation and followed by Feature extraction & classification, which happens as a one-stage process by CNN. The resultant data of the training, feature extraction, classification is designed to get stored as .pb5 and .h5 model files. This will help during the actual application time - by fetching the data from the stored files (.pb5 and .h5) instead of fetching from the beginning stage. The validation image (taken on real time basis) is tested in the designed system to detect & identify the plant diseases and estimate the calorie and nutritional facts of fruits, vegetables, raw & cooked foods.
This developed CNN based system is integrated with latest technologies YOLO v5 and Stremlit framework for improved results and to provide a user-friendly experience. Improvisation achieved with YOLO v5, by identifying the number of objects in the given validation image, and also recognize the irrelevant objects in the image.









, Claims:WE CLAIMS

1. An efficient and cost-effective deep learning-based model which detects & identifies the type of plant diseases in both Fruits & Vegetables using CNN technology.
2. A single model which is capable to detect and classify multiple types of fruits and vegetables & in its clustered form.
3. An integrated model which combines CNN & YOLO v5 techniques - to analyse real-time customized data towards the objective of identifying plant diseases, estimation of calorie & nutritional facts in fruits, vegetables, raw and cooked foods.
4. A single model which identifies and recognizes both raw foods and cooked foods - towards the objective as claimed in Claim #2.
5. A single model which analyzes both raw foods and cooked foods, and give an estimation on amount of calorie.
6. In addition to the claim #5, the model also estimates the macro nutrients such as Carbohydrates, proteins, fats and fiber content in a single process.
7. An efficient and enhanced model with consideration to the Indian health habits, the datasets given as the input to the YOLO v5 is customized with inclusion of Indian foods & Indian snacks.

Documents