Party Bus Mission DevOps & Ops Support Levels
Customer Support Models
Below are the support models that Mission DevOps (MDO) uses to quickly address Product Team needs. Please understand, Party Bus Mission DevOps is NOT a 24/7 support organization. We do not support Mission Application 24/7 requirements. The Mission DevOps team provides support between the hours of 0800 CST-1800 CST M-F only. Any support outside business hours will be addressed next business day.
DISCLAIMER
Support will only be given if demonstrated that the issue is with the Party Bus platform which does not include application team managed resources or Iron Bank images.
Tiered Support
Tiered support escalates issues until they reach personnel with the technical expertise necessary to resolve them. The Party Bus Operations (PB-Ops) team ensures tickets are immediately escalated when needed and worked in a timely manner when placed in a new queue, preventing customers from experiencing longer queue times and ultimately creating a better support experience.
Swarming Support
Swarming IT support functions by allowing engineers to collaborate on an issue until it's resolved- issues that are best suited for swarming support are complex and involve multiple steps. Swarming support teams proactively watch for incoming tickets that may best be resolved in this manner, then work to resolve them as quickly as possible. The goal is to acquire the correct resources and to resolve the issue as quickly as possible in conjunction with fellow team members.
Hybrid Tiered/Swarm Support Model
A Hybrid Tiered/Swarm support model allows for tickets to be assigned at a Tiered level and swarmed when needed. As issues leave the Tier 1 or Tier 2 support level, they are assigned to the Tier 3 or Tier 4 support level and worked solo or in collaboration as needed until resolution.
Tiered MDO DevOps Support Levels
| Support Level | Function | Swarm? | Team | Escalation | Outage Response Time | SLA Time to Initial Response Working Hours | SLA Time to Resolved (After Initial Response) Working Hours |
|---|---|---|---|---|---|---|---|
| Tier 0/Automation and Self-Serve | Users have identified an issue and conduct independent research and information retrieval (i.e., self-help). Users enter search criteria in the Platform One Support Hub to identify potential resolutions to their issue. | No | Self Service | If the User is unable to identify a resolution to their issue with indepedent research, the User should open a support ticket or reference the End-To-End Guide. | 0800 CST-1800 CST, M-F | N/A | N/A |
| Tier 1 | After a User has opened a ticket, this first level of support addresses the issue with generalist knowledge. Support at this level has no in-depth technical knowledge. | No | Razor Crest | Ticket is opened in the Platform One Hub. | 0800 CST-1800 CST, M-F | N/A | N/A |
| Tier 2 | If Tier 1 is unable to resolve or the ticket is triaged as Tier 2, Tier 2 serves as the second level of support, and assists in resolving the issue. This may require teaming up with developers to research and resolve the issue. NOTE: Swarming may be required at this level, but it is not typical. | No | Razor Crest/Mission DevOps Team | - Elevated from Tier 1 or Ticket is triaged immediately as Tier 2. - May need to swarm at this level. | Immediate 0800 CST-1800 CST, M-F | Staging < 16 hours Production < 8 hours | Staging < 24 hours Production < 24 hours |
| Tier 3/CICD Changes, Manifest Changes, Overlay Changes | If Tiers 1 and 2 are unable to resolve or the ticket is triaged as Tier 3, this third level of support comprises expert product and service-level personnel, including developers and other product specialists. This tier may require teaming up with developers to research and resolve the issue. Subject to internal prioritization. | Yes | Mission DevOps/Party Bus Operations Team | -Elevated from Tier 2 or Ticket is triaged immediately as Tier 3. -This is where the Mission DevOps and Operations teams swarm on an issue. | < 1 hour PB Ops Support 0800 CST-1800 CST M-F | < 16 hours | Depends on the type of issue. May be a new feature request subject to prioritization by the development team. No guarantee on timeframe for resolution, or resolution at all. |
| Tier 4/Architectural Changes Beyond CICD, Vendor Intervention | Tier 4 comprises internal expert support, or external vendor support that goes far beyond the norm. If an issue has reached this tier it is because it requires access to the underlying architecture of the platform. | Yes | Party Bus Operations Team | -Tier 4 is the last line of defense. -External Vendor Support or external to PB support | < 5 days | Depends on the type of issue. May be a new feature request subject to prioritization by the development team. | No guarantee on timeframe for resolution, or resolution at all. |
| Tier 5/Beyond Help Desk - Request for Feature | Brand new integrations to the service catalog / infrastructure Strategies to enable and actual applications fall under new Feature Request. Tickets that come into this queue can be denied due to internal deliberation and / or cyber security posture. | N/A | Party Bus Leadership | Upvoting places the feature request at a higher priority because it shows multiple customers would like to see this feature implemented. | TBD 7am to 5pm EST | < ? hours | < ? hours |
| SCA MDO | Trufflehog, Twistlock, and Anchore findings. | N/A | Mission DevOps Team | -Triaged immediately. -All submitted issues are resolved same day. -SCA issues that can be resolved in 5-15 minutes. | Immediate 0800 CST-1800 CST M-F | < 8 hours | < 3 hours |
| SCA CAT | Fortify, Sonarqube, Dependency Check, and Pen Test findings | N/A | Cyber Applications Team | 0800 CST-1800 CST M-F | < 24 hours | < 8+ hours |
DISCLAIMER
Time to resolution may vary based upon issues/blockers found during the research process.
NOTE
Frequent software package upgrades for Big Bang tools may cause outages or service interruptions. In consideration of our c-ATO, we are required to be an N-1 software version organization, meaning we must maintain the latest software version and packages of our Cyber scanning tools at all times. P1 provides announcements via multiple communication channels to notify users of these upgrades and potential outages, and include a detailed README of all changes. While Big Bang upgrades are rigorously tested, not all issues are captured. This may cause an outage that requires Party Bus to respond within the Tier 3 SLA timeframe.
Triage
Triage managers perform the initial review of tickets and place them into the appropriate queues. The primary intent is for the triage manager to get a feel for the types and quantity of tickets received, and add automation to better service the customer need.
Pipeline/Deployment Support
The MDO team supports IL2/IL4/IL5 deploys only. SIPR/JWICS pipeline support may differ from the below table. Please contact the Classified Ops team through their Mattermost channel .
Party Bus Customers are expected to keep their code base up to date by actively running production pipelines at least 1x per week.
| Request Type | Escalation | SLA Time to Initial Response | SLA Time to Resolved (After Initial Response) | Notes |
|---|---|---|---|---|
| Pipeline Request | MDO PM PB OPs PM | 1-2 weeks | 1-2 weeks 0800 CST-1800 CST, M-F | Includes staging deploy configurations |
| Auxiliary Deployment Request | MDO PM PB OPs PM | 1-2 weeks | 1-2 weeks 0800 CST-1800 CST, M-F | For large applications time to resolve will vary greatly |
| Production Deploy Request | MDO PM PB OPs PM | 1-3 days | 1-2 days 0800 CST-1800 CST, M-F | Requires signed CTF letter |
| CTF Renewal/Extension Request | MDO PM PB OPs PM | 1-3 days | 1-2 days 0800 CST-1800 CST, M-F | Requires signed CTF letter |
DISCLAIMER
Time to resolution may vary based upon issues/blockers found during the pipeline creation process.
Support is provided 0800 to 1800 CST.
Compute and Storage Thresholds
This table defines the compute and storage limitations and thresholds for Party Bus Application Teams. If these are insufficient for an application, Party Bus customers will be subject to an audit and potential fees.
| Type | Threshold | Notes |
|---|---|---|
| Staging/Prod Deployment Pod Resource defaults | limit cpu: .5 limit memory: 256mb request cpu: .1 request memory: 64 mb | |
| Redis/ELK/MongoDB PVC | limit cpu: .5 limit memory: 256mb request cpu: .1 request memory: 64MB Storage limit: 50GB ELK (MDO PROD IL2) | No High Availability. |
| S3 | Total storage: No limit Single file size limit: | Total storage: Typically a 1-2 TB |
| Pipeline Artifacts | 2000MB max | Expires in 7 days. Used to store build artifacts and small files. |
| Pipeline Jobs | cpu: 4 cores limit memory: 10GB request cpu: 0.25 cores request memory: 1GB | All pipeline jobs. |
| Postgres/MySQL RDS | 200GB max |
Product Team Support Levels
This table defines the Support Levels Party Bus Application Teams can choose when onboarding or renewing. If the support hours are insufficient for an application team, Party Bus customers will be subject to an audit and potential fees.
| Level | Hour Limit | Additional Benefits | Notes |
|---|---|---|---|
| Platinum | 451 | Platform One partner Priority 1 support response (automatic priority escalation over other tickets of the same type) 1:1 support and conference call assistance provided as needed | Required if application is identified as Complexity 4 |
| Gold | 290 | With PB Leadership approval, 1:1 support and conference call assistance (case by case) | Default recommendation if application is identified as Complexity 3 |
| Silver | 173 | With PB Leadership approval, an initial 1:1 conference call assistance | Default recommendation if application is identified as Complexity 2 Minimum support level if application requires dedicated database |
| Bronze | 32 | Helpdesk ticket support only No 1:1 conference call assistance | Default recommendation if application is identified as Complexity 1 |