Appearance

Powered by Platform One

MDO Announcements ​

The Mission DevOps team will announce changes concerning pipelines and/or deployments affecting customers in this space. Be sure to watch this page for notifications!

Announcements are updated here AND in Mattermost's MDO Help Channel .Please also check the active and archived Mattermost Notification Bot .

26 August 2025: Clamav Scanning for Image Build Pipelines ​

Attention Party Bus Customers! To continue to improve the efficiency and security of image build pipelines, you will see a new job called clamav-scan that will perform an AV scan and block the pipeline if malware findings are found within your container image. In the past, we have used the functionality built into Anchore to perform AV scans; however, this will now be handled by the clamav-scan job in your image build pipelines.

If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

9 July 2025: Gitlab SAST and Merge Request Workflows updates ​

As announced previously, Party Bus is moving towards Gitlab SAST (Advanced SAST and semgrep) for their preferred SAST scanner over Fortify. As such, we have higlighted some important workflow changes including the following:

  • Merge request pipelines become a standard across all teams
  • Scans are enforced using Merge request policies to catch new findings
  • No more direct pushes to master branch

Announcement here for more info

SAST Scan Vulnerability Reports ​

We've enabled Gitlab SAST scans across all pipelines. You'll start seeing these security scan results in two places:

Merge Request Workflow Changes ​

What's Changed ​

You may have noticed some behavioral changes around merge request creation and pipeline execution. These updates are part of our first phase to improve the merge request workflow.

Key Benefits ​

  • No more branch protection required: Teams no longer need to protect their branches to run pipelines
  • Simplified process: Simply open a Merge Request to trigger a pipeline

Important Notes ​

  • The number of stages that run during merge request pipelines may change as we continue our discovery process
  • Pipeline configurations are subject to updates based on ongoing improvements

How to Re-run Merge Request Pipelines ​

  1. Navigate to the Merge Request in the UI
  2. Select the Pipelines tab
  3. Click Run Pipeline

Known Issues ​

Current Limitations: These changes have caused some adverse effects with:

  • Protected variables functionality (Not to be confused for Masked variables). If running an MR pipeline with an unprotected branch, the pipeline will no longer be able to access protected variables. This is intentional and project level variables can be un-protected as needed by your team. We have already removed the instance-level protected variables as necessary
  • Manual pipeline runs for open MRs (see above for how to remediate this)

We're actively investigating these issues and working on fixes as they arise.

Next Steps ​

  • In the coming days/weeks you may see a Security Bot message in your Merge Requests to show the new policies in effect for enforcing SAST. These policies are non-enforcing and are simply meant to be informational. No action is required, but it may be helpful to start understanding and/or remediating findings
  • Example Messages from Gitlab Security Bot are as follows: securitybot mr message
  • Whitelisting workflows are still in the works. As this is a completely new process, we are taking the steps to help teams transition as smoothly as possible

13 June 2025: Fixed inconsistent exclusion of base image vulns in Twistlock ​

Attention Party Bus Customers. We have fixed issues with how Twistlock vulnerability findings are attributed to the base image and reported in anchore-twistlock-results job.

What this does - reduces need for help desk tickets ​

  • Default gatecheck should no longer incorrectly flag findings from the base image
  • Base image findings will be correctly reported for RBD analysis

Additional Details ​

Problem ​

Previously, the pipelines depended on VAT data or Twistlock to correctly identify the base image and the associated findings, but often the base image scan results were not up to date enough to use for this. Additionally, Twistlock settings were causing some findings to not be reported.

Solution ​

  • Base image scan occurs in its own job earlier in pipeline
  • Twistlock reporting no longer excludes base image findings
  • Instead, anchore-twistlock-results job uses base image scan results for whitelisting base image findings for default gatecheck

19 May 2025: Risk-based Deployments (RBD) has launched! ​

Attention Party Bus Customers. We have implemented a major change to the anchore-twistlock-results job PASS/FAIL logic.

  • We have added flexibility to the passing pipeline criteria for evaluating vulnerability findings
  • This does not directly affect CtF requirements nor does it change customer workflows

What does RBD do? ​

  • Reduces the need for CVE exception tickets and gets your changes deployed to Staging and Prod faster!
  • Findings affecting software already deployed to Production will no longer block future release deployments
  • Provides enhanced data reporting to facilitate risk reviews and remediation efforts

Additional Details ​

Your results job will now PASS if the software is considered less risky than your previous Production release OR if it passes the previous default gate checks.

Examples of Passing Job Criteria:

  • (NEW) No findings are added (only carried over from Production release)
  • (NEW) Risk score has improved
  • No findings for Twistlock or Anchore default gatechecks

Questions? ​

If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

19 Mar 2025: Adjusting Twistlock Severity Filter ​

Attention Party Bus Customers. As part of our effort to streamline the vulnerability remediation process for customer images, we are now filtering out some additional severities from triggering a Twistlock gatecheck failure in the anchore-twistlock-results.

These are some additional low severity types that have minimal impact on the deployed app's security and will help speed up deployment by filtering these out.

If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

13 Feb 2025: Moving Anchore and Twistlock Gatechecks to their own Job ​

Attention Party Bus Customers. As part of our effort to streamline the vulnerability remediation process for customer images, we have moved the vulnerability gatechecks from the anchore-scan and twistlock-scan jobs to a new job called anchore-twistlock-results.

Moving forward, you should expect the following:

  • The anchore-scan and twistlock-scan jobs should only fail as a result of an abnormal pipeline failure (as opposed to a failure resulting in identified vulnerabilities)
  • Any pipeline failure due to identified CVEs will take place in the anchore-twistlock-results job
  • The details on the vulnerabilities identified and any gatecheck failures due to present vulnerabilities will now be present in the anchore-twistlock-results job
  • There are now collapsable sections (that are collapsed by default) in the anchore-twistlock-results job that will provide details on the identified vulnerabilities
  • At the end of the anchore-twistlock-results job, it will inform you of any gatecheck failures do to vulnerabilities identified by anchore and/or twistlock.
  • If your pipeline has anchore or twistlock disabled (rare) please put in a ticket with MDO to modify your pipeline accordingly

If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

23 Jan 2025: Switching from Buildah to Podman for Image Builds ​

Attention Party Bus Customers. To keep our pipelines secure and up to date, we have made the following change to our customer image build pipelines:

We have changed from using Buildah to Podman as our tool for building images. For the most part these tools have like for like functionality and capabilities but you may notice some difference in the base image detection logic. If you feel these changes have affected your image builds or image scans please open a ticket.

If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

09 Sep 2024: Streamlining Prod Image Scans ​

Attention Party Bus Customers. As continued work towards our effort to better assess risk within our pipelines, you will notice the following functionalities have been modified:

  • Moving forward you should no longer see Prod Image Scan pipelines created by the MDO bot running in your project. These pipelines have been streamlined to only gather and store CVE data. This change enhances our ability to gather the latest vulnerability information on a released image and compare it against the findings for current images created in your application image build pipelines.

Please Note: If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

05 Sep 2024: ArgoCD Pod Restart in production ​

As requested, we have now rolled out the ability to restart pods in production by utilizing ArgoCD. As a part of this feature request, we have decided that it is more advantageous to be able to restart workloads, not just delete pods to trigger a new rollout. The delete capabilities are still present but we are also adding the ability to Restart Deployments and StatefulSets. See this [how-to]("/docs/Party Bus/Mission DevOps (MDO)/How Tos/ArgoCD/_index.md#restarting-workloads-in-argocd") on more information. Thank you!

03 Sep 2024: Update Image for Maven and Gradle CI templates ​

Attention Party Bus Customers– as mentioned two weeks ago, we have updated default java images for the maven and gradle pipeline-templates from JDK 11 to JDK 17.

  • registry1.dso.mil/ironbank/opensource/gradle/gradle-jdk11:8.5 to registry1.dso.mil/ironbank/opensource/gradle/gradle-jdk17:7.6
  • registry1.dso.mil/ironbank/opensource/maven/maven-openjdk-11:3.8.6 to registry1.dso.mil/ironbank/opensource/maven/maven-openjdk-17:3.9.3-amazoncorretto-17

22 Aug 2024: Storing release artifacts in the image registry ​

Attention Party Bus Customers. As continued work towards our effort to better assess risk within our pipelines, you will notice the following functionalities have been added:

  • Two new jobs have been added upload-to-registry and fetch-release-artifacts
    • These jobs facilitate the storage and retrieval of the artifacts used to compare changes between releases
  • Specific artifacts used to compare releases will be stored in your project's container image registry under boe-artifacts

Please Note: If the either of the added job fail for some reason, this will not block you from being able to perform a prod release or block the CTF process. If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

15 Aug 2024: Update Image for Maven and Gradle CI templates ​

Attention Party Bus Customers, we are upgrading default java images for the maven and gradle pipeline-templates in 2 weeks, 30 AUG 2024.

We are currently using JDK11 and we will be moving to JDK17.

  • registry1.dso.mil/ironbank/opensource/gradle/gradle-jdk11:8.5 to registry1.dso.mil/ironbank/opensource/gradle/gradle-jdk17:7.6
  • registry1.dso.mil/ironbank/opensource/maven/maven-openjdk-11:3.8.6 to registry1.dso.mil/ironbank/opensource/maven/maven-openjdk-17:3.9.3-amazoncorretto-17

If you are still using JDK11, please note that your pipelines may potentially break if you are not prepared.

If you are having any issues, please submit a help desk ticket .

30 July 2024: Body of Evidence Release ​

Attention Party Bus Customers. As continued work towards our effort to better assessing risk within our pipelines, you will notice the following functionalities have been added:

A new job has been added for pipelines that run on the default branch of a project called boe-release. This is a manual/optional job you can run to allow you to create a Body of Evidence (BoE) release. These releases will show up under Deploy -> Releases for your project, just like prod releases. Creating a BoE release will allow you to easily view the summary-report and package-cross-reference, as well as other relevant asset links. It will also package up all the reports generated in the pipeline into a BoE tar archive. These reports will help enable assessment of risk prior to a prod release.

Please Note: If the generate-summary-report job fails for some reason, this will cause the boe-release job to fail if run. However, it will not block you from being able to perform a prod release. Failure of either the generate-summary-report or boe-release jobs will not block the CTF process. If you have any feedback or questions, feel free to let us know in the Party Bus support channel . Thank you!

29 July 2024: Auxiliary Deployments ​

Teams can request to have another branch deploy to staging and then to production. This is accomplished by having a mirror push changes to another gitlab project. More details in the documentation

Once consulting with BAMs, you may submit a request to use another branch

24 July 2024: Gitlab Scan Policies ​

Starting this week, Party Bus is rolling out some scan policies that will inject the Party Bus pipelines with a custom semgrep scan to detect usage of the compromised polyfill[.]io CDN (content delivery network). These scans are implemented using some new Gitlab Ultimate features, namely the Scan Execution Policies. If these scans affect your pipelines in any way or you run into the error: Pipeline filtered out by workflow rules, please open a ticket and we will address accordingly. Thank you!

17 July 2024: Party Bus Risk Based Pipelines ​

Attention Party Bus Customers. You may have noticed some additional behaviors and jobs added to your pipelines. Party Bus is gathering information and re-assessing the way that we look at risk within our pipelines. You will notice the following functionalities have been added, including but not limited to:

  • The addition of Prod and Staging comparison tools to evaluate how secure your staging deployments are compared to what is deployed in production
  • The gathering of artifacts including but not limited to SBOMs, pipeline commit information, scanning tool versions. Most notably, new jobs run after your staging and production releases to push SBOMs to our internal software inventory and vulnerability dashboard, DefectDojo
  • A list of CVEs within your project, prioritized by some experimental factors (will be refined in future)
  • Lastly, An executive summary of the items mentioned above, which can be found in the releases area of your project after a production deploy

Currently, there are no required actions or gate checks within the new stages/steps within the pipelines as it is all information gathering at this time. We aim to provide a better picture of health for yourselves and our team. Additionally, we will continue announce more changes as we add and refine this process. If you have any feedback, feel free to let us know in the Party Bus support channel . Thank you!

20 Jun 2024 : after_script commands moving to script ​

GitLab is changing the behavior of the after_script keyword in v17. It will run for cancelled jobs, which is not our desired behavior. We have been using it to run additional commands after the "main" script block. We will be moving that logic to the script block. The after_script block allowed commands to fail and not affect the status of the job, whereas commands in the script block will. Pipelines run after the change may now be blocked because of previously ignored errors. Please address issues in your repo as needed and/or open a help desk ticket if the pipeline needs to be adjusted.

Common issues found:

  • .vue converter => Fortify is unable to process .vue files; we have scripts to convert them into .js files; make sure they are compatible with your pipeline
  • executing commands on files that do not exist => the files were either moved or the commands are no longer needed; please let us know how it needs to be adjusted

04 Mar 2024 : pgAdmin is now available! ​

PGAdmin can be used to manage your data without having to go through the help desk. In staging, you'll be able to fully manage your data. This means creating tables, inserting data, updating rows, and even deleting data. You can even have PGAdmin in production! Even though it's locked down to read-only, it remains as an important asset in investigating your most obscure/elusive production bugs.

Request to have PGAdmin today by submitting a help desk ticket !

Note, please submit one ticket per environment. For example, if you have il2-staging and il4-prod, you would submit two tickets.

Please refer to our pgadmin documentation for more details.

28 Apr 2023 : New Pipeline Features ​

Hello, we will be merging new features into pipeline-templates within the next two weeks. We will update this channel as plans and dates start to solidify. Stay tuned!

The new features are:

  • Switching from a stage-based pipeline to a needs-based pipeline. As long as pre-requisite jobs are completed, the job in question will run. With that in mind, we simplified our stages to: .pre, ci, staging, and prod. You may also switch the pipeline view from "Stage" to "Job Dependencies".

  • Cache-artifact redesign. Currently, cache is used to speed up jobs and to pass artifacts down the pipeline to subsequent jobs. With this change, cache is solely used to speed up jobs and artifacts are used to pass artifacts to subsequent jobs. This change has been made to align with gitlab's intended design. If you are using a cpp, golang, dotnet, gradle, or maven template, you will not see a big difference. However, if you are using any other template, you will notice that the "build" job will no longer disappear on you.

  • Build jobs are now renamed to align closer to the main command that is actually running.

    • dotnet - dotnet-publish
    • cpp - build-cpp
    • golang - go-build
    • gradle - gradlew-assemble
    • maven - mvn-package
    • npm - npm-ci
    • php - composer-install-prod / composer-install-dev
    • python - pip-install
    • ruby - bundle-install
    • yarn - yarn-install

Thank you! We appreciate your cooperation.

MDO Team

after

16 Nov 2022 : Automatic Pipeline Runs ​

Starting November 28th Cyber and Mission DevOps teams will perform daily scans of applications that are deployed. They will be checking all deployed images for the application to see if they have had a pipeline run of their master branch within the last week. If there has not been one, we will automatically kick off a pipeline run. The reason for this is because new CVEs/Vulnerabilities are discovered weekly and if your application is not going through our security scans, your application could have an exposed attack vector. Please address any new findings that may come to light from these pipeline runs.

View changes to this P1 policy .

08 Aug 2022 : Anchore ​

As a part of an effort to add more layers of vulnerability detection, Party Bus will be rolling out Anchore image vulnerability scanning today for deployment pipelines on 8/8 in IL2/IL4/IL5. This new feature will be added to the current image scan stage and will perform similar duties as the Twistlock scan. Thus, teams may see both scans fail in the scan-image stage for the same vulnerabilities. If this happens and the team has proper justification for whitelisting, then please open a single ticket to address vulnerabilities in both scans. That being said, this will currently be a beta feature and will be allowed to fail for the time being.

Additionally, There will also be an anchore job that generates a Software Bill of Materials (SBOM) as an artifact in order to give a more complete picture of what software is installed on each image. Note that This stage is not a gate check and should always pass unless technical issues arise

19 Jul 2022 : Dependency Check Upgrade to Version 7.1.1 ​

Dependency Check Update to 7.1.1 on Tuesday 7/19/2022 Fix bash globbing bug that failed to handle wildcards properly such as

**/*.jar

Impact:

some pipelines may exhibit additional dependency check findings. These findings most likely already existed in the project but were obscured by the globbing bug. some pipelines may observe some false positives eliminated. See the release notes for version 7.1.1

https://github.com/jeremylong/DependencyCheck/releases

28 Jun 2022 : CtF Check ​

PB-MDO will be releasing a new version ctf_check that will validate CTF letter info during production deployment jobs. This new version is backwards compatible and we expect zero impact if you are already able to deploy with the current system. Please submit a help disk ticket if you encounter issues.

20 Jun 2022 : Python pylint Job ​

Party Bus Mission DevOps will be pushing a change to the python pylint job. This change will go into affect on Monday 6/20/2022.

This job will run the pylint command with no additional switches. Application teams will manage their own .pylintrc files in their repository to manage pylint settings.

Documentation for configuring pylint.

04 May 2022 : ClamAV ​

Hello folks 👋 If your app allows users to upload files, Cyber requires that they be scanned. To satisfy this requirement, we have implemented a multitenant instance of clamav for product teams to use. The service is available in IL2/4/5 in staging and production. Please see the following guide for implementation instructions, https://confluence.il2.dso.mil/display/P1MDOHD/HowTo+-+ClamAV+-+Customer+Guide.

04 May 2022 : Deploy to Staging ​

Party Bus Mission DevOps is planning to push a change at 1700 EST/1500 MST that involves a change in configuration for deploying product team containers. This change will be rolled out to IL2, IL4, and IL5 and will involve utilizing different tokens for manipulating the kustomization deployment manifests. These tokens are already in place and therefore projects should not see changes in their deploy to staging or release stages. This also should not affect the argo portion of the deploy stage. That being said if you experience issues with deploy-to-staging and/or releases please create a help desk ticket and we will treat it as priority.

17 Mar 2022 : Commit Signing ​

@here REMINDER: code.ilx.dso.mil will start enforcing code signing March 18th. Please open your Code Sign verification ticket soon. Here are the CaC and GPG setup articles. Please NOTE: Setup in your IDE is not supported by the MDO team. Please test at the command line.

09 Mar 2022 Use of harden-nodejs in image builds ​

It has come to our attention that teams are still using the registry.ilx.dso.mil harden-nodejs image and possibly other base images within registry.ilx.dso.mil that were NOT created specifically for their team. These images were created as a stopgap and are no longer necessary for use. If you are using any of the harden images, please switch to using the most recent ironbank image instead. see this doc for guidance:

https://confluence.il2.dso.mil/pages/viewpage.action?spaceKey=P1PARTYBUS&title=Party+Bus+-+Project+Settings+Enforcement

02 Mar 2022: Code Signing ​

On Friday March 18th, we will be turning on a setting in GitLab that will require teams to sign their commits. Commits can be signed with a GPG Key or your CAC. However, if your projects are in IL4 and IL5 you should use your CAC as we will restrict signatures to CACs only, in the future. Commits that aren't signed will be rejected through a pre-receive hook. Please follow our guides on setting up code signing: HowTo - GitLab - Code Signing with GPG and HowTo - GitLab - Code Signing with CAC .

28 Jan 2022: Fortify Gate-check ​

Tomorrow (Jan 28th), MDO will be pushing out an update to Fortify's gate-check script. This update may cause some Fortify jobs that are currently passing to fail. We will be enabling allow_failure for the Fortify job in order to give product teams time to remediate the findings.

Please take time to review your pipelines and the Fortify Job in order to determine if your team will need to address any findings. Fortify job failures will be allowed until Feb 4th.

14 Jan 2022: Turning Off allow_failure Setting ​

IMPORTANT ANNOUNCEMENT: Allowing failures in staging pipelines will no longer be permitted effective 4 Feb 2022. We are turning off the allow_failures = true setting. If your application currently allows failures in staging, your code will no longer be allowed to deploy.

This means that in order to deploy to staging, unit tests and lint need to pass and all compliance checks and SCA findings will have to be resolved before deploying to staging.

The only exception to this are jobs that have been recently introduced to the pipeline and are set to allow failure at a global level. These jobs are: find unauthorized, project configs, lint for gradle and maven only, e2e tests, and pen test.

This is a P1 security decision, endorsed by our ISSO, supporting the cATO.

05 Jan 2022: Default Runner Build Directory ​

Hello everyone. MDO will be updating the default build directory for runner jobs to be /builds/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME. This will standardize the path across all teams and across all impact levels. The GitLab default was causing issues between jobs for some teams that relied on absolute paths.

What this means for you:

  • if your project is using relative paths, you likely won't notice anything
  • if your project uses absolute paths in any of your jobs (usually sonarqube), we've probably had to implement a workaround to allow continuous operation of your pipeline. We will be removing these workarounds. If any of your jobs are suddenly failing because it is unable to locate files, please let us know
  • you can see what the the build directory is set to for any runner job at the beginning of the log, it looks like: Initialized empty Git repository in <BUILD DIR>

Relevant links:

https://docs.gitlab.com/ee/ci/runners/configure_runners.html#custom-build-directorieshttps://jira.il2.dso.mil/browse/PBMD-2632

17 Nov 2021: Pipeline Changes - Compliance Check ​

Hi everybody, we have renamed the "Dockerfile Scan" stage of the pipeline to "Compliance Check" and we have added a new job under it called project-configs.

This job will verify that your repository's Gitlab CI/CD configuration is set up according to our project settings requirements. Additionally in the coming weeks we will be auditing all repositories in Party Bus Gitlab to verify that pipelines are configured according to these requirements.

A couple notes about our CI/CD requirements:

.gitlab-ci.yml files are not permitted in repositories in Party Bus All pipelines in Party Bus Gitlab must be managed by the MDO team.

Relevant documentation:

16 Nov 2021: Introducing Gradle Lint Support ​

Hello everyone, Gradle pipelines now have a lint job. This job works by running a Gradle task ./gradlew lintGradle. This job will be allowed to fail at a global level for two weeks as we test it out.

We have documented this in these two articles:

25 Oct 2021: Docker Image Requirements Changes ​

Hello everyone! We are making a change to the "Find Unauthorized" job under the "Dockerfile Scan" stage of the pipeline.

This change adds a new enforcement check to verify that your Dockerfiles are only using approved base images:

  • Images in the pipeline-templates repository container registry
  • Ironbank

If you are not using images from one of these two sources please update your Dockerfile to reference them.

We have documented these requirements in the following places:

As a final note - we are not enforcing this job at this time and it is allowed to fail. In the future this job will not be allowed to fail; we will provide at least two weeks of notice before that time.

28 Sep 2021: Twistlock Scan Changes ​

Hi all, we are slightly changing the way the Twistlock scan stage of the pipeline works:

  • Exceptions for CVE findings in base images will now be granted automatically
  • CVE findings should now only occur for items added to the image in your Dockerfile

What this means for you:

  • If you see Twistlock findings and they are NOT from your app, try using a newer image
  • If you see Twistlock findings from your app and you believe they are false-positives, please open a help desk ticket.

Relevant links:

16 Sep 2021: Python Template Changes ​

Swapping Pipeline Image

We are switching out the pipeline image. When this happens you will need to clear runner caches, (CI/CD > Pipelines > Clear Runner Caches).

  • From registry.il2.dso.mil/platform-one/devops/pipeline-templates/ubi8-python3.8:8.1.
  • To an image based on registry1.dso.mil/ironbank/opensource/python/python39:v3.9.6.

Dependency Check

We changed how python dependencies are scanned in dependency check. Instead of scanning your python packages directory, we will scan a generated requirements.txt that will include all dependencies (main + transient) with versions specified for each. We have found that this method yields less false-positives and more accurate findings. The generated requirements.txt will be created in the build job and saved into artifacts for you to download if you want.

Lint

We upgraded pylint to 2.10.2. With that, a new feature has been added that allows you to ignore directories. E.g. ignore-paths=^./.cache/.*$ would ignore .cache in scans using a .pylintrc or a switch.

  • Note, in the near future we plan to create new python template using pipenv. With that change, we plan to stop using pylint environment variables like PYLINT_EXCLUSIONS and PYLINT_EXTRA_SWITCHES. The new pipenv template will follow the npm template and let you choose your linter. If plan to keep using pylint, I suggest switching to using a .pylintrc as soon as possible and move away from the environment variables.

Unit Tests

We introduced a UNITTEST_EXCLUSIONS environment variable to exclude certain directories from running tests. By default, this is set to .cache/,/usr/lib/_,test*.py.

  • The pipenv changes that were mentioned for lint also apply to unit tests.

Build

The build job will only run if there are changes to your requirements.txt (change in your dependencies) similar to the npm template.

We plan to merge this change on Friday EOD. I plan to announce a reminder beforehand.

03 Sep 2021: Build Job and Cache ​

If you do not see the build job when you trigger a pipeline it is because build is meant to download your dependencies and store them in cache to be used for subsequent pipeline runs. The cache does not expire. However, it can be cleared manually through CI/CD > Pipelines > Clear Runner Caches. If it is cleared you will run into issues in the pipeline if build does not run again. The build job triggers only when there are changes to your package-lock.json or if the pipeline is triggered manually through CI/CD > Pipelines > Run Pipeline. TL;DR: if you clear the cache, run the pipeline manually to rebuild your dependency cache; CI/CD > Pipelines > Clear Runner Caches then CI/CD > Pipelines > Run Pipeline.

This is only applicable to the npm and yarn template, but will eventually reflect all pipelines.

19 Aug 2021: Nodejs Pipeline Image Update ​

TL;DR

Platform One is changing the Nodejs docker image used in the pipeline's build, lint, and unit test jobs from Docker Hub's node:slim image to Iron Bank's Nodejs16 image.

When will this Occur? MON 23 AUG 2021

What to Expect? Platform One has successfully tested the new image on multiple projects and we do not expect there to be a lot of fallout from the change. However, if you do experience an issue please create a Jira Service Desk ticket and we'll take care of it as quickly as we can.

Who is Affected? This change is for any node projects built with either npm or yarn.

Why is this Happening? Platform One is committed to constantly improving its security posture. That is the motivation behind the switch from the public Docker Hub image to one from the Iron Bank, which manages a repository of digitally signed, binary container images that have been hardened and accredited for DoD-wide use across classifications.

05 Aug 2021: Dockerfile Linting Change Justification ​

Hey everyone, I wanted to add a note in here to address our reasoning behind the recent Dockerfile pipeline changes.

We added the "dockerfile lint" job for the following reasons:

Our C-ATO requires that product teams use Ironbank images. These images have been vetted for vulnerabilities and hardened by the IB team to reduce potential attack surface. Enforcing Dockerfile best practices leads to more efficient use of P1 resources and reduces chances of bugs. Read more here: https://docs.docker.com/develop/develop-images/dockerfile_best-practices/

We added the "find unauthorized" job for these reasons:

Pipelines already compile code and download dependencies during the build stage, reusing these artifacts saves runner resources and time and will speed up pipelines across Party Bus. Downloading and installing external packages breaks the hardnening work done by Ironbank, potentially exposing the cluster to new vulnerabilities. Code and dependencies are scanned for security during the SCA stage of the pipeline. Reinstalling or rebuilding these during the Docker build means we can no longer guarantee the code and artifacts have been scanned. While we scan the images with Twistlock, it is not a static code analysis tool. Identifying unauthorized commands early allows teams time to update their Dockerfiles before the CTF process. 02 Aug 2021: New "Find Unauthorized" Pipeline Job

We are introducing a new Dockerfile Scan stage into the pipeline which will have the dockerfile lint job that is currently in .pre and a new find unauthorized job.

Allow failure has been removed from the dockerfile lint job as of this morning. This runs hadolint and will set to fail for issues that are a warning or error. It will also check if you are pulling images from an approved registry. Approved registries are:

*registry.il2.dso.mil *registry.il4.dso.mil *registry.il5.dso.mil

The new pipeline job, find unauthorized will scan your Dockerfile for any unauthorized commands. If it finds any of the commands or keywords listed within a command it will flag it as an issue. We will allow failure for now to give you all a chance to address any issues found before we start to enforce it.

Please note that the job will fail if the Dockerfile is not in the root of your repo.

Please visit [Pipeline Basics - Find Unauthorized Stage(https://confluence.il2.dso.mil/display/P1MDOHD/HowTo+-+GitLab+-+Pipeline+Basics)] for the list of commands/keywords, how to resolve these type of issues, and instructions on how to run this job locally. docker file scan

29 Jul 2021: Hadolint Guidance ​

The hadolint warning warning: Last USER should not be root is a valid finding. You can resolve the warning by switching back from root at the end of every stage. There is a note about this in the docker lint stage documentation . Also a reminder that this stage will be enforced starting Monday.

22 Jul 2021: Notice of Docker Lint Stage Enforcement ​

Starting on August 1st we will be enforcing the Docker lint stage of the pipeline. This means that we will not allow failures in this stage for all applications in all impact levels, including both staging-only apps and production apps.

This stage currently is set to enforce that your Dockerfiles only pull from approved images as well as other basic Dockerfile best practices.

What this means for you:

  • If your application is currently passing the Docker lint stage and uses approved images, there are no changes for you.
  • If you are using an unapproved image, you will need to update your Dockerfile to use an approved image .
  • If you have previously been given an exception to use a different image than in the approved list, please request an exception for this enforcement through a help desk ticket .

16 Jul 2021: IL2 E2E Stage Functional ​

The E2E stage is now functional in IL2 for apps that run cypress tests. There is also new functionality in the Deploy Staging pipeline stage such that the job will wait until ArgoCD has deployed the new version of your app before completing. This will ensure the E2E stage runs tests against the new version of your app.

What this means for you:

  • If your pipelines and staging application are in IL2 and your repository has cypress tests you can expect these to now run correctly.
  • If your app uses a different e2e testing framework than cypress, please submit a feature request .
  • If your pipelines are in IL2 but your staging application is in IL4 or above, E2E tests will not work for you. Let us know and we can disable the argo wait condition for your app.
  • If you are not in IL2, there are no changes for you yet. We are still working on IL4 and IL5 implementations.
view pipeline basics documentation on confluence .

If you have any questions or concerns feel free to ask in this channel or through a help desk ticket .

Powered By Platform One