Automation Of Deep Learning With Jenkins

PROJECT

 Automation Of a Deep Learning Model

 

Task Overview - Creating a Deep Learning Model to predict the objects of an image using Docker. This model will train automatically, when the Docker image is launched. Jenkins will automatically pull the repo when developer commits. Jenkins will automatically deploy the code and start training the model using CNN, Jenkins will automatically start the container required for processing. It will predict the accuracy of trained model. If the accuracy is less than 80%, then tweak the machine learning model the machine learning model architecture, retrain the model and send the notification on mail to Developer. A extra job is given to Jenkins to monitor the app container, If the container fails due to any reason Jenkins will automatically start the container again.   

Tools Used –

• ·        Jenkins
• ·        Git
• ·        Github
• ·        Redhat Linux
• ·        Docker

Description -

• Docker Image- Build a Docker image with Tensorflow and Sklearn installed in it. When we launch the image it will automatically start to train the model.
• Jenkins Job- A Job Chain of 4 jobs is created using Build Pipeline in Jenkins.

1.      Job1- Pull the Github Repo automatically when a Developer commits.

2.      Job2- Jenkins will automatically start the Container, deploy the code and with start training the model using CNN architecture.

3.      Job3- Model will predict accuracy. If accuracy is less than 80%, then tweak the Machine Learning Model, Retrain the model and Predict the accuracy again.

4.      Job4- This job is to send the notification on the mail.

 

 

·        Jenkins Job 5- A extra Job Job5 is given to the Jenkins for monitor. If the Container fails due to an reason, This Job will start the Container again And also send the E-mail notification.

 

Building The Project

 

Creating The Docker File –

        Tensorflow Dockerfile-

               FROM centos:latest

                 RUN yum install python36 –y

                 RUN python3 –m pip install –upgrade pip

                 RUN pip3 install –upgrade setuptools

                

                 RUN yum install  -y epel-release

                 RUN yum groupinstall “development tools” –y

                 RUN yum install –y python3 devel

 

                 RUN pip3 install keras

                 RUN pip3 install numpy

                 RUN pip3 install pandas

                 RUN pip3 install matplotlib

                 RUN pip3 install pillow

                 RUN pip3 install opencv-python

                 RUN pip3 install –upgrade tensorflow

                 ENTRYPOINT [“python3”]

                 CMD [“/mycode/cnn.py”]

 

 

               Sklearn DockerFile-

               FROM centos:7

               RUN yum install python36 –y

               RUN python3 –m pip install –upgrade pip

               RUN pip install –upgrade setuptools

               RUN pip3 install pandas

               RUN pip3 install numpy

               RUN pip3 install sklearn

               RUN pip3 install joblib

               RUN pip3 install matplotlib

 

Jobs In Jenkins –

o   JOB 1:  The task for job 1 is to pull the repo from github. For this we have to write following code.

sudo cp –v –r –f * /root/mlops1

o   JOB 2: The task for job 2 is to start the container and deploy the code. For this we have to write following code in Jenkins.

If sudo docker ps | grep jet

then

        echo “already running”

        docker rm o1

fi

sudo docker run –i –name jet –v /root/mlops1 alex43/ubuntu-deeplearning-env:v1.0

o   JOB 3: The task for job 3 is to train the model and predict the accuracy metrics. If accuracy is less than 80%, then tweak the machine learning model architecture, retrain the model and predict accuracy again.

accuracy= $(sudo cat /root/mlops1/mycode/output.txt)

acc= $(echo “($result+0.5)/1” | bc)

res=90

if [ $acc –gt $res ]

then

        echo “ACCURACY IS : $acc”

else

        if sudo docker ps | grep jas

        then

               docker rm jas

        fi

        sudo docker run –i –name jas –v alex43/ubuntu-deeplearning-env:v1.0fi

o   Job 4: The task for job 4  is to send a email notification to the developer.

o   Job 5: This is a extra job given to Jenkins to monitor the container. If due to any reason container fails, this job will start the container again.