Appearance

Powered by Platform One

E2E Testing ​

There are currently no requirements from the MDO team as to what this stage needs to do, as long as it passes. The number of tests and scope of tests is up to the application team to decide but should cover a reasonable amount of functionality so as to verify application integrity. However, Certificate to Field (CtF) review may have certain requirements that must be met prior to CtF authorization.

E2E CtF Requirements ​

  1. All projects/pipelines must perform e2e tests.

Per Project Requirements

ALL** pipelines/projects should perform these tests. The PartyBus team understand that this creates some confusion in terms of testing on backend pipelines vs testing on frontend pipelines. Our current guideline is, at a bare minimum to define one suite of tests per frontend and then run them in all pipelines. We are working to make this process less inefficient and will keep all teams posted on if this changes

  1. e2e tests must perform basic functionality tests and also cover any requirements in SDElements that are not met by other stages of the pipeline. If you are not familiar with SDElements or have not gone through the CtF Process, please open a ticket here (link) to start the process.

  2. All tests are performed on frontend, but should encompass any backend functionality as well.

    a. Essentially you are testing your "Application" as a whole and should keep this in mind for your tests. See notes for more on what this looks like in a microservice/pipeline environment.

  3. Party Bus supports Cypress testing by default, but this is not a requirement.

Functionality and Technical Guidelines ​

  • Currently for cypress tests we define a location where screenshots/videos from your pipeline will be archived. this location is cypress/screenshots. Please use this as an output directory in order to be able to look at the results from your e2e tests in a visual format. This will better help you diagnose any issues with your e2e testing in the P1 environment.

  • E2E testing will need to be performed against your staging environment. We use the variable ZAP_URL to define your staging address. use this as your baseurl in cypress.

  • If your application parses the P1 auth header/JWT in order to provide extra levels of authorization/RBAC for your app, then you will need to define a test JWT for e2e testing and you will need that JWT be able to authenticate properly. Refer to this link for more information.

E2E Gateway

Party Bus uses a separate gateway to bypass keycloak in this stage, so at no point should your app be getting any keycloak errors or redirects back. Or be blocked by keycloak in general. Please open a ticket if this is an issue.

  • As stated above, PartyBus supports Cypress testing by default, but if your project requires anything other than Cypress, you will need to create a help desk ticket. Note that Party Bus does not provide support for anything other than Cypress and Pytest so if your application requires another method of testing, you will need to have fully fleshed out your tests and have them be functional against a staging environment using the guidelines above.

E2E Support

The PartyBus team does not provide guidance on how to define cypress tests or any other type of e2e testing. It is on each product team to learn cypress or any other e2e testing suite and define those tests on their own. We can assist if you are having issues testing from the pipelines, but you will need to provide some proof that it is a PLATFORM issue and not a CODE issue.

Cross-IL Setups

E2E testing across Impact Levels (ILs) is not currently supported. The staging environment must be in the same IL as the source code. In this situation, E2E testing will need to be done locally and results captured and passed along to Cyber.

Cypress ​

You will need a script in package.json called test:e2e-ci that defines how the E2E test is invoked. Generally the script in package.json will look as follows:

json
"test:e2e-ci": "npx cypress run --browser chrome --config-file cypress/config/pipeline.config.js"

https://docs.cypress.io/guides/end-to-end-testing/writing-your-first-end-to-end-test

Customers manage their own Cypress version. In your Gitlab repository, you may create a CI/CD variable titled "CYPRESS_VERSION" and set the value to the desired Cypress image tag (ie. set the value to 14.3.0). The Cypress image used in the pipeline is from Ironbank and called "cypress-included". You may find the available versions in Ironbank via this link.

Pytest ​

You will need to set E2E_PYTEST to your E2E entrypoint. We assume pytests/e2e_pytest.py by default.

Data Isolation ​

After the migration from the use of separate e2e and staging clusters, there is no longer any data isolation for e2e. All data is the same since the e2e and staging cluster are one in the same.

WARNING

AT THIS TIME, UNSUPPORTED DATABASES AND THEIR ISOLATION ARE THE RESPONSIBILITY OF THE PRODUCT TEAM.

FAQs ​

How does the JWT work for E2E tests? ​

Since auth-service and Keycloak are not present in the E2E gateway, requests will hit your app without a JWT. It is recommended that your tests add a mock JWT header.

View example implementation.

What URL should my pipeline e2e tests use? ​

Your E2E tests in the staging pipelines should use the staging URL of your application.

Appendix ​

For information on how this job is implemented, visit HowTo - GitLab - Pipeline Basics.

For announcements pertaining to E2E testing jobs, visit MDOE - Announcements .

Powered By Platform One