Fear, Stress and Chaos: What Does the 3PAO Want From Me?

Fear, Stress and Chaos: What Does the 3PAO Want From Me?

FedRAMP can be challenging. All the paperwork, monitoring and logging has the potential to break even the most cohesive cloud service provider (CSP). And when preparing for a 3PAO assessment, numerous stressful questions often arise:

What can a CSP do to prepare for the assessment?

What information will the 3PAO request?

Will the 3PAO provide a questionnaire detailing the artifacts the CSP is expected to develop?

The test cases workbook contains all the controls (grouped by family) and helps to drive consistency for assessments performed by 3PAOs. CSPs may use the same workbook to prepare for testing by using the test cases to understand the assessment procedures and documenting control status prior to the formal verification and validation of the security controls.

How does this work? Let’s use the test case information from control AC-6(5) as an example.

In this example, the FedRAMP test cases define the methods the 3PAO is expected to execute in order to verify using examine, test and interview. As defined in NIST 800-53A, the examine method is the process of reviewing, inspecting, observing, studying or analyzing one or more assessment objects (i.e., specifications, mechanisms or activities). The purpose of the examine method is to facilitate assessor understanding, achieve clarification or obtain evidence. The interview method is the process of holding discussions with individuals or groups of individuals within an organization to facilitate assessor understanding, achieve clarification or obtain evidence. The test method is the process of exercising one or more assessment objects (i.e., activities or mechanisms) under specified conditions to compare actual with expected behavior.

By understanding the definitions, the CSP is able to ascertain what the 3PAO will expect during the assessment. Using AC-6(5) as the example, the CSP can expect:

Method for 3PAO

Expected from CSP

Examine

Policy that explains how non-privileged access for no security functionsProcedure that addresses least privilege

Walkthrough (test/demonstration) of how least privilege functions as described in the system security plan (SSP)

Interview

Discussion with individual/role identified in the SSP regarding the implementation of the controlInterview response should align to documented process and results from test

With the examine, test and interview criteria, the CSP can use the information to prepare for the assessment. For example, the CSP may decide to use the worksheet internally and enter the information directly into the test cases spreadsheet as shown below:

Another method Lunarline has seen CSPs implement is to use a ticketing system or CDN to track the status of the controls, collect artifacts and assign responsibilities. A ticket/page is created for each security control, and personnel attach the required evidence directly to the ticket/page. The page not only provides a convenient place to store information, but it’s also used as part of continuous monitoring to provide evidence to the assessor that policies, process and procedures are reviewed on a periodic basis. The ticket system or CDN automatically tracks changes and maintains historical records. Another neat trick we’ve seen is the ticket system automatically generating tickets as part of continuous monitoring to notify personnel it’s time to review a log, execute a scan or conduct training. The ticket is not closed until the task is complete.

Ultimately, beyond the mandatory templates provided by the project management office (PMO), FedRAMP provides a CSP with the flexibility to collect, track and monitor security controls using a method that best suits the CSP. The key to success is ensuring the CSP is able effectively meet the criteria defined in the 3PAO assessment test cases.

In addition to focusing on the test cases, CSPs should also ensure the control status and implementation explanation statements within the SSP align to the expected assessment results. If a control is fully implemented, then there should be a clear description of compliance in the SSP control statement, and all test case methods should be met. If a control is partially implemented, the CSP can assist the 3PAO during the assessment by ensuring the implementation description in the SSP contains sufficient information to describe the portion of the control that’s integrated into the system and the portion of the control not (yet) implemented. For planned controls, the CSP should ensure sufficient information is available for the 3PAO to understand safeguards/countermeasures in order to determine the overall level of risk based on likelihood and impact.

So to return to the original questions:

What can a CSP do to prepare for the assessment?Lunarline recommends that the CSP thoroughly review the security assessment test cases provided by the FedRAMP PMO. The SSP control status should be accurate and implementation statements provide sufficient details to support the expected results from the test cases.

What information will the 3PAO request?The 3PAO will request evidence to meet the examine, test and interview methods defined in the test cases.

Will the 3PAO provide a questionnaire detailing the artifacts the CSP is expected to develop?Not all 3PAOs provide questionnaires to the CSP. Therefore, it is recommended the CSP use the test cases workbook to determine the list of artifacts expected to be submitted as part of the assessment. Beyond the mandatory PMO-provided templates, CSPs have the flexibility to use any template or format that best suits it.

How will the 3PAO assess my systems?The 3PAO will follow the FedRAMP test cases. As the system expert, it is recommended that the CSP develop internal methods for “testing” that provide evidence the system is functioning as described in the SSP.

Keep in mind, a 3PAO’s objective is not to scare a CSP. 3PAOs are not spies. Also, the rumor is not true: 3PAOs do not achieve a higher ranking based on the total number of findings during an assessment. The 3PAO’s role is to verify and validate the information contained within the CSP’s SSP. Results from the assessment are based on examinations, tests and interviews — nothing more, nothing less. CSPs that allocate sufficient time to fully comprehend, internally review, and document the desired results from the tests cases prior to the formal assessment, have the highest degree of likelihood to have a positive experience during the event. The test cases worksheet is the primary source of information that describes how the 3PAO will conduct the assessment. Internally documenting the artifacts, test methods and assigning roles in spreadsheet or ticket systems can greatly assist CSPs in organizing the assessment supporting artifacts and evidence.