6.1. CI/CD Devtools Ecosystem

6.1.1. Agility

• Pair Programming in PyCharm: https://www.jetbrains.com/help/pycharm/code-with-me.html

• Further Reading: https://dev.astrotech.io/agile/index.html

6.1.2. Ecosystem

• Further Reading: https://dev.astrotech.io/summary/index.html

6.1.3. Version Control System

• Git

• Git Flow

• Github

• Bitbucket

• GitLab

• GitOps: https://www.gitops.tech

• FluxCD: https://github.com/fluxcd/flux

• Further Reading: https://dev.astrotech.io/git/index.html

6.1.4. Virtualization

• Docker

• LXC - Linux Containers

• OCI - Open Container Initiative

• Kubernetes

• Containerd

• OpenShift

• Open Stack

• Amazon EKS, ECS

• Further Reading: https://dev.astrotech.io/docker/index.html

6.1.5. Continuous Integration / Delivery

• Jenkins

• Github Actions

• Bitbucket Pipelines

• CircleCI

• Travis

• GitLab

• Further Reading: https://dev.astrotech.io/jenkins/index.html

6.1.6. Quality Assurance

• SonarQube

• SonarLint 1

• SonarScanner

• SonarCloud

• Coverage

• PEP-8

• PyLint

• Black

• Further Reading: https://dev.astrotech.io/sonarqube/index.html

• Further Reading: https://python.astrotech.io/devops/ci-cd/tools.html#static-analysis

• Further Reading: https://python.astrotech.io/devops/ci-cd/code-style.html

• Further Reading: https://python.astrotech.io/devops/ci-cd/coverage.html

• Further Reading: https://python.astrotech.io/devops/ci-cd/static-analysis.html

6.1.7. Issue Tracker

• Jira

• Gitlab

• Github issues

• Jira Integration: https://jira.astrotech.io/end-user/automation.html

• Further Reading: https://dev.astrotech.io/jira/index.html

6.1.8. SSH

• Further Reading: https://dev.astrotech.io/linux/index.html

6.1.9. Testing

• Further Reading: https://test.astrotech.io

6.1.10. Mutation Testing

6.1.11. BDD Testing

• Lettuce: http://lettuce.it/index.html

• Cucumber: https://cucumber.io

• Behave: https://behave.readthedocs.io/en/stable/tutorial.html

6.1.12. Load Testing

• Gatling: https://gatling.io

• JMeter: https://jmeter.apache.org

6.1.13. Testing UI

• Selenium: https://www.selenium.dev

6.1.14. Testing microservices

• Further Reading: https://arch.astrotech.io

• Source: https://martinfowler.com/articles/microservice-testing/

6.1.15. Provisioning

• Ansible

• Puppet

• Chef

• Salt, SaltStack

• Vagrant

• Further Reading: https://dev.astrotech.io/puppet/index.html

• Further Reading: https://dev.astrotech.io/ansible/index.html

• Further Reading: https://dev.astrotech.io/vagrant/index.html

6.1.16. Assignments

curl https://get.docker.com |sudo sh
sudo usermod -aG docker ubuntu   # requires logout

Jenkins:

docker network create ecosystem
mkdir -p /home/jenkins
chmod 777 /home/jenkins
chmod 777 /var/run/docker.sock
ln -s /home/jenkins /var/jenkins_home

docker run \
    --name jenkins \
    --detach \
    --rm \
    --network ecosystem \
    --publish 8080:8080 \
    --volume /home/jenkins:/var/jenkins_home \
    --volume /var/run/docker.sock:/var/run/docker.sock \
    jenkins/jenkins:alpine

SonarQube:

docker network create ecosystem
docker volume create --name sonarqube_data
docker volume create --name sonarqube_extensions
docker volume create --name sonarqube_logs

docker run \
    --name sonarqube \
    --detach \
    --rm \
    --network ecosystem \
    --publish 9000:9000 \
    --volume sonarqube_data:/opt/sonarqube/data \
    --volume sonarqube_logs:/opt/sonarqube/logs \
    --volume sonarqube_extensions:/opt/sonarqube/extensions \
    sonarqube

SonarScanner:

• sonar-project.properties

• Further Reading: https://dev.astrotech.io/sonarqube/sonarscanner.html

• Further Reading: https://python.astrotech.io/devops/ci-cd/static-analysis.html

## Sonar Server
sonar.host.url=http://sonarqube:9000/
sonar.login=admin
sonar.password=admin

## About Project
sonar.projectKey=myproject
sonar.projectName=myproject
sonar.sourceEncoding=UTF-8

## SonarScanner Config
sonar.verbose=false
sonar.log.level=INFO
sonar.showProfiling=false
sonar.projectBaseDir=/usr/src/
sonar.working.directory=/tmp/

## Build Breaker
sonar.buildbreaker.skip=false
sonar.buildbreaker.queryInterval=10000
sonar.buildbreaker.queryMaxAttempts=1000

## Debugging
# sonar.verbose=true
# sonar.log.level=DEBUG
# sonar.showProfiling=true
# sonar.scanner.dumpToFile=/tmp/sonar-project.properties

## Python
sonar.language=py
sonar.sources=.
sonar.inclusions=**/*.py
sonar.exclusions=**/migrations/**,**/*.pyc,**/__pycache__/**

docker run --rm --network ecosystem -v $(pwd):/usr/src sonarsource/sonar-scanner-cli

Docker Registry:

docker network create ecosystem
mkdir -p /home/registry
chmod 777 /home/registry

docker run \
    --name registry \
    --detach \
    --rm \
    --network ecosystem \
    --publish 5000:5000 \
    --volume /home/registry:/var/lib/registry \
    registry:2

Tests:

Code 6.52. test-unit.sh

#!/bin/sh

cd example-py-unittest
python3 -m unittest

Code 6.53. test-functional.sh

#!/bin/sh

cd example-py-doctest/
python3 -m doctest -v doctests/*

Code 6.54. test-integration.sh

#!/bin/sh

pip install -r requirements.txt
cd example-py-pytest/
python3 -m pytest

Code 6.55. test-static.sh

#!/bin/sh

docker run --rm --net ecosystem -v $(pwd):/usr/src sonarsource/sonar-scanner-cli

Code 6.56. make-artifact.sh

#!/bin/sh

REGISTRY='localhost:5000'
NAME='myapp'
VERSION="$(git log -1 --format='%h')"

IMAGE="$REGISTRY/$NAME:$VERSION"

docker build . -t $IMAGE
docker push $IMAGE
docker rmi $IMAGE

6.1.17. References

1

https://www.sonarlint.org