Pages

Showing posts with label Jenkins. Show all posts
Showing posts with label Jenkins. Show all posts

Thursday, December 17, 2020

Jenkins Pipeline to update Docker Image

 We basically pull the Docker image from DEV ECR and Docker push to the platform & Region's ECR and then update the Deployment file and apply the kubernetes deployment.

Keywords : Pipeline, Multiple Condition(SwitchCase), e-mail

pipeline {

  agent {

    label 'appjenkins-slave'

  }

    options {

        ansiColor('xterm')

        timestamps ()

    }

    parameters {

    choice( choices: ['preprod-us' , 'preprod-eu' , 'preprod-de'], description: 'Region', name: 'EnV')

    string(name: 'DockerVersion', defaultValue: 'v1.1', description:'Docker image Version')

string(name: 'AccountNumber', defaultValue: '061320014550', description:'Account Number')

    }


stages {

stage('Docker Pull') {

steps {

withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'app-terraform@506077034049@APPortal-DEV', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) {

sh label: '', script: '''#!/bin/bash -e

echo $(/usr/local/bin/aws ecr get-authorization-token --region us-east-1 --output text --query 'authorizationData[].authorizationToken' | base64 -d | cut -d: -f2) | docker login -u AWS https://506077034049.dkr.ecr.us-east-1.amazonaws.com --password-stdin

docker pull 506077034049.dkr.ecr.us-east-1.amazonaws.com/ssp/backend:${DockerVersion}

'''

}

}

}

stage('Analyze Platform') {

parallel {

stage ("PREPROD-US") {

when { expression { params.EnV == 'preprod-us' } }

steps {

withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'app-terraform@061320014550@APPortal-PREPROD', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) {

sh label: '', script: '''#!/bin/bash -e

SCM_URI="${AccountNumber}.dkr.ecr.us-east-1.amazonaws.com"

docker tag 506077034049.dkr.ecr.us-east-1.amazonaws.com/ssp/backend:${DockerVersion} ${SCM_URI}/ssp/backend:${DockerVersion}

echo $(/usr/local/bin/aws ecr get-authorization-token --region us-east-1 --output text --query 'authorizationData[].authorizationToken' | base64 -d | cut -d: -f2) | docker login -u AWS https://${SCM_URI} --password-stdin

docker push ${SCM_URI}/ssp/backend

sudo -u ssp-${EnV} /bin/bash -c "id; pwd"

                         echo; echo Update IMAGE URL; echo '======================'

sudo grep image: /home/ssp-${EnV}/sspbackend.yaml | cut -d":" -f 3

IMAGEVER=`sudo grep image: /home/ssp-${EnV}/sspbackend.yaml | cut -d":" -f 3`

echo "Existing Version: "$IMAGEVER

  echo "Need to Replace with: "$DockerVersion

sudo sed -i "s/$IMAGEVER/$DockerVersion/g" /home/ssp-${EnV}/sspbackend.yaml

sudo cat /home/ssp-${EnV}/sspbackend.yaml

sudo -u ssp-${EnV} /usr/local/bin/kubectl apply -f /home/ssp-${EnV}/sspbackend.yaml

'''

}

}

}

stage ("PREPROD-EU") {

when { expression { params.EnV == 'preprod-eu' } }

steps {

withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'app-terraform@061320014550@APPortal-PREPROD', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) {

sh label: '', script: '''#!/bin/bash -e

SCM_URI="${AccountNumber}.dkr.ecr.eu-west-1.amazonaws.com"

docker tag 506077034049.dkr.ecr.us-east-1.amazonaws.com/ssp/backend:${DockerVersion} ${SCM_URI}/ssp/backend:${DockerVersion}

echo $(/usr/local/bin/aws ecr get-authorization-token --region eu-west-1 --output text --query 'authorizationData[].authorizationToken' | base64 -d | cut -d: -f2) | docker login -u AWS https://${SCM_URI} --password-stdin

docker push ${SCM_URI}/ssp/backend

sudo -u ssp-${EnV} /bin/bash -c "id; pwd"

                         echo; echo Update IMAGE URL; echo '======================'

sudo grep image: /home/ssp-${EnV}/sspbackend.yaml | cut -d":" -f 3

IMAGEVER=`sudo grep image: /home/ssp-${EnV}/sspbackend.yaml | cut -d":" -f 3`

echo "Existing Version: "$IMAGEVER

  echo "Need to Replace with: "$DockerVersion

sudo sed -i "s/$IMAGEVER/$DockerVersion/g" /home/ssp-${EnV}/sspbackend.yaml

sudo cat /home/ssp-${EnV}/sspbackend.yaml

sudo -u ssp-${EnV} /usr/local/bin/kubectl apply -f /home/ssp-${EnV}/sspbackend.yaml

'''

}

}

}

stage ("PREPROD-DE") {

when { expression { params.EnV == 'preprod-de' } }

steps {

withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', accessKeyVariable: 'AWS_ACCESS_KEY_ID', credentialsId: 'app-terraform@061320014550@APPortal-PREPROD', secretKeyVariable: 'AWS_SECRET_ACCESS_KEY']]) {

sh label: '', script: '''#!/bin/bash -e

SCM_URI="${AccountNumber}.dkr.ecr.eu-central-1.amazonaws.com"

docker tag 506077034049.dkr.ecr.us-east-1.amazonaws.com/ssp/backend:${DockerVersion} ${SCM_URI}/ssp/backend:${DockerVersion}

echo $(/usr/local/bin/aws ecr get-authorization-token --region eu-central-1 --output text --query 'authorizationData[].authorizationToken' | base64 -d | cut -d: -f2) | docker login -u AWS https://${SCM_URI} --password-stdin

docker push ${SCM_URI}/ssp/backend

sudo -u ssp-${EnV} /bin/bash -c "id; pwd"

                         echo; echo Update IMAGE URL; echo '======================'

sudo grep image: /home/ssp-${EnV}/sspbackend.yaml | cut -d":" -f 3

IMAGEVER=`sudo grep image: /home/ssp-${EnV}/sspbackend.yaml | cut -d":" -f 3`

echo "Existing Version: "$IMAGEVER

  echo "Need to Replace with: "$DockerVersion

sudo sed -i "s/$IMAGEVER/$DockerVersion/g" /home/ssp-${EnV}/sspbackend.yaml

sudo cat /home/ssp-${EnV}/sspbackend.yaml

  sudo -u ssp-${EnV} /usr/local/bin/kubectl apply -f /home/ssp-${EnV}/sspbackend.yaml

'''

}

}

}

}

}

stage("Mail Notification") {

steps {

emailext body: '''Deployed the ${DockerVersion} Docker image for SSP-${EnV}-API \n Build Status : ${currentBuild.currentResult} \n Job name: ${JOB_NAME} \n Build Number: ${BUILD_NUMBER} \n More info at: ${BUILD_URL}''', recipientProviders: [[$class: 'RequesterRecipientProvider'], [$class: 'DevelopersRecipientProvider']], replyTo: 'tilak.bhandari@hidglobal.com, naveen.kumar@hidglobal.com, laxmappa.malligawad@hidglobal.com, Sangeeth.VeluthedathSahadevan@hidglobal.com, spurtiganga.hulamani@hidglobal.com, ashwin.nagaraja@hidglobal.com', subject: '${DEFAULT_SUBJECT}', mimeType: 'text/html', to: 'joelandrews.jamesselvam@hidglobal.com, tilak.bhandari@hidglobal.com, naveen.kumar@hidglobal.com, laxmappa.malligawad@hidglobal.com, Sangeeth.VeluthedathSahadevan@hidglobal.com, spurtiganga.hulamani@hidglobal.com, ashwin.nagaraja@hidglobal.com'

}

}

}

}

Thursday, July 23, 2020

AWS: Upload Docker Image to ECR Repo

To upload the Docker Image to ECR Policy:
ENV:
projectName=sismanager
region=us-east-1
serviceName=application-management
awsAccount=arn:aws:ecs:us-east-1:396773889961
clusterName=sismanager-cluster-preeval
environment=preeval
awsServiceName=${projectName}-${serviceName}-service-preeval
awsTaskName=${projectName}-${serviceName}-task-preeval
latestTag=preevallatest

Shell-1:
#!/bin/bash
MANIFEST=$(aws ecr batch-get-image --repository-name ${environment}/${projectName}/${serviceName} --image-ids imageTag=${release_version} --query images[].imageManifest --output text --region ${region})
aws ecr put-image --repository-name  ${environment}/${projectName}/${serviceName} --image-tag ${latestTag} --image-manifest "$MANIFEST" --region ${region} || error=true
if [ $error ]
then
    echo "[OUTPUT] - ${latestTag} is already applied to the image"
else
    echo "[OUTPUT] - ${latestTag} is applied to the image"
fi
   
aws ecr describe-images --repository-name ${environment}/${projectName}/${serviceName} --region ${region}

Shell-2: - update
#!/bin/bash
#check whether task exists
#if task not exists, deploy new code and ensure new code is running
#if task exits, get the current task name, deploy new code and ensure new code is running and kill the old task

startTask ()
{
    echo "Starting Tasks"
    taskRevision=`aws ecs describe-task-definition --task-definition ${awsTaskName} --region ${region} | jq .taskDefinition.revision `
    if [ $? -eq '0' ]
    then
        echo "[OUTPUT] - Revision is ${taskRevision} "
echo ${taskRevision}
aws ecs update-service --cluster ${clusterName} --region ${region} --service ${awsServiceName} --task-definition ${awsTaskName}:${taskRevision} --desired-count 1 --force-new-deployment
    else
    echo "[OUTPUT] - Task revision not found. So unable to start the task"
        exit 1
fi
}

stopTask ()
{
    aws ecs stop-task --cluster ${clusterName} --task ${trimGetTask}
    if [ $? -eq '0' ]
    then
        echo "[OUTPUT] - Stop task command executed successfully"
        checkTask=`aws ecs describe-tasks --cluster ${clusterName} --tasks ${trimGetTask} |jq '.tasks[].desiredStatus'`
        echo "[OUTPUT] - Task status is ${checkTask}"
        trimCheckTask=`echo $checkTask | tr -d \"`
        echo "[OUTPUT] - After slashing task name is ${trimCheckTask}"
        if [ "$trimCheckTask" == "STOPPED" ]
        then
            echo "[OUTPUT] - Task has been successfully stopped"
            exit 0
        else
            echo "[ERROR] - Issue while stopping task"
            exit 1
        fi
    else
        echo "[ERROR] - Stop task command didnt executed successfully."
        exit 1
    fi
}


echo "[OUTPUT] - Checking if any tasks exists already"
taskCount=`aws ecs list-tasks --cluster ${clusterName} --service-name ${awsAccount}:service/${awsServiceName} |jq length`
echo "[OUTPUT] - Task count is $taskCount"
if [ "$taskCount" -eq '1' ]
then
    echo "[OUTPUT] - Task is running"
    getTask=`aws ecs list-tasks --cluster ${clusterName} --service-name ${awsAccount}:service/${awsServiceName} |jq '.taskArns[0]'`
    echo "[OUTPUT] - Task name is ${getTask}"
    trimGetTask=`echo $getTask | tr -d \"`
    echo "[OUTPUT] - After slashing task name is ${trimGetTask}"
    startTask
    #stopTask
else
    echo "[OUTPUT] - No task is running. So starting task"
    startTask
fi



Wednesday, January 31, 2018

Create and configure Linux Partition on VM

Lets discuss how can we add a new partition on Linux VM.
1. First edit the settings on the VM and add harddisk,

2. Run fdisk -l. you can see a new /dev/sdb with what space you provided. it is still the Raw harddisk partition. We need to partition it and make it usable.

3. Assume that /dev/sdb is the new hdd we had, run the following commands,
[root@LINUX] # fdisk /dev/sdb
it'll ask a prompt, we should give n as option to create a new partition and mention the first and last cylinder for its size. then give w to write the partition table.

4. To verify the partition, run the fdisk -l, now it'll show the partition. assume /dev/sdb1 is the partition which we created now. now we should make the partition readable to linux. run the following command,
[root@LINUX] # mkfs -t ext4 /dev/sdb1


5. Now mount the partition manually to some partition. in my case i am going to create a separate var partition for my jenkins, so i ran the following commands,
[root@LINUX] #  mv /var /var-OLD
[root@LINUX] # mkdir /var
[root@LINUX] # mount /dev/sdb1 /var

6. To add this mount permantenly please add the entry to /etc/fstab file
[root@LINUX] # vi /etc/fstab

/dev/sdb1 /var ext4 defaults 0 0

7. Reboot your machine, you are all set.

FYI,
incase for jenkins, if you move /var to /var-OLD and create new partition. please do this step, else you'll not able to connect to the VM through SSH/Putty client. and start the sshd service.

[root@LINUX] # cp -fr /var-OLD/empty /var
[root@LINUX] # service sshd status

Wednesday, December 14, 2016

editable email - print variables

In Jenkins using editable e-mail plugin, we can't directly print the value of the variables,



In this scenario, i get CHANGES_SINCE_LAST_BUILD considered as a text instead of variable, to overcome this issue and incase if you want to replace value for the variable, then we should use.

{ENV,var="CHANGES_SINCE_LAST_BUILD"}



Wednesday, August 7, 2013

Windows 2008 R2 slave failed to connect via DCOM

Scenario:

          If we configure Windows 2008 R2 node to Jenkins, it fails to connect to the node.

Solution 1 :
      Try connect the node through JNLP, it'll connect.

Solution 2:
       If we want to connect through Windows service, please follow the steps.

  1. If you see the below error,

        Access is denied. [0x00000005]
 at org.jinterop.dcom.core.JIComServer.init(JIComServer.java:542)
 at org.jinterop.dcom.core.JIComServer.initialise(JIComServer.java:458)
 at org.jinterop.dcom.core.JIComServer.<init>(JIComServer.java:427)
 at org.jvnet.hudson.wmi.WMI.connect(WMI.java:41)
 at hudson.os.windows.ManagedWindowsServiceLauncher.launch(ManagedWindowsServiceLauncher.java:107)
 at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:170)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:619)
Caused by: rpc.FaultException: Received fault. (unknown)
 at rpc.ConnectionOrientedEndpoint.call(ConnectionOrientedEndpoint.java:142)
 at rpc.Stub.call(Stub.java:112)
 at org.jinterop.dcom.core.JIComServer.init(JIComServer.java:538)
  1. Turned off the firewall (this could be configured correctly to be safer, but I didn't care since its in a firewalled "safe" part of the net)
  2. Launch 'regedit.exe' as 'Administrator' 
  3. Find the following registry key: 'HKEY_CLASSES_ROOT\CLSID{76A64158-CB41-11D1-8B02-00600806D9B6}' refer here
  4. Right click and select 'Permissions' 
  5. Change owner to administrators group. 
  6. Change permissions for administrators group. Grant Full Control. 
  7. Restart Remote Registry Service 
  8. Restart the Windows machine.

Jenkins Failed to locate Cygwin installation. Is Cygwin installed?

Scenario:
 I configured a new Windows 64 bit node in Jenkins and when i try to execute a shell command from it, i got the below mentioned error.


FATAL: command execution failedhudson.util.IOException2: Failed to locate Cygwin installation. Is Cygwin installed? at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.getCygwinRoot(CygpathLauncherDecorator.java:122) at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.call(CygpathLauncherDecorator.java:127) at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.call(CygpathLauncherDecorator.java:112) at hudson.remoting.UserRequest.perform(UserRequest.java:118) at hudson.remoting.UserRequest.perform(UserRequest.java:48) at hudson.remoting.Request$2.run(Request.java:287) at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:72) at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source)Caused by: hudson.util.jna.JnaException: Win32 error: 2 - null at hudson.util.jna.RegistryKey.check(RegistryKey.java:124) at hudson.util.jna.RegistryKey.open(RegistryKey.java:223) at hudson.util.jna.RegistryKey.openReadonly(RegistryKey.java:218) at hudson.plugins.cygpath.CygpathLauncherDecorator$GetCygpathTask.getCygwinRoot(CygpathLauncherDecorator.java:115) ... 11 more

Solution 1:
 Just add the "Shell" on the first line or your Execute Shell part. it may resolve the issue.
#!/bin/bash
Best Solution is don't mention any shell, so that it'll run the script whether on CMD / SHELL.

Solution 2:
 Check if sshd service is running if not please follow the instructions.

Solution 3:
 If sshd service is not configured as a service in windows, then follow the instructions to create the service.

Solution 4:
 If sshd service is configured but starting saying "service started and stopped as no process is using the service". then check the below,
  1. Find the Cygwin log path, in my case C:\cygwin\var\log\sshd.log. if you observed,

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0711 for '/etc/ssh_host_rsa_key' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /etc/ssh_host_rsa_key
Could not load host key: /etc/ssh_host_rsa_key

2. Then change the permission of the files like this, no write / now execute. Then i started the service, it came up.
$ chmod 600 ssh_host_*
-rw-------   1 cyg_server     None    988 Aug  6 14:40 ssh_host_key
-rw-r--r--+  1 cyg_server     None    652 Aug  6 14:40 ssh_host_key.pub
-rw-------   1 cyg_server     None   1675 Aug  6 14:40 ssh_host_rsa_key
-rw-r--r--+  1 cyg_server     None    407 Aug  6 14:40 ssh_host_rsa_key.pub
-rw-------   1 cyg_server     None    672 Aug  6 14:40 ssh_host_dsa_key
-rw-r--r--+  1 cyg_server     None    615 Aug  6 14:40 ssh_host_dsa_key.pub