Western Governors University reduces testing time from 96 hours to 5-6 with Postman

The free version of the Postman API Platform had initially met WGU's needs but as the engineering teams grew, managing usage at scale became a challenge. No reporting existed for developer productivity, making it difficult to track metrics. Furthermore, license allocation was a manual process that involved downloading and auditing user data once a month.

When WGU's Postman journey began, testing was a highly manual process. Each request was triggered, reviewed, and recorded manually. For each ticket, basic information was added manually. Not only did this make record tracking and trend spotting very difficult, but it often resulted in the full test suite not being run as the process was so tedious. ServiceNow tests would take upwards of 30 minutes, and multiple team testing could take up to four hours.

One of the key tests undertaken involved the quality engineering (QE) team simulating vast amounts of online traffic in an attempt to crash WGU's servers. This load testing was done with JMeter, but issues could not be logged when a system went down, so critical data was lost. WGU had considered using a live environment for load testing which would have resulted in additional network stress being placed on WGU's single-sign-on (SSO) tool, Ping Identity. Load testing needed to be improved, and performance testing needed to be implemented.

Finally, API documentation was sparse, making onboarding new hires or team members laborious. The lack of API documentation resulted in the need for extra meetings to ensure a universal understanding of each API. Due to data silos and tool sprawl, there was no cohesive reporting structure, and API standards were not enforceable.

Ultimately, WGU's API governance model needed to be streamlined as it involved Postman, ServiceNow, and third-party integrations. This also resulted in collaboration being a challenge.

With ambitious goals in mind, WGU needed an API platform that could address a broad spectrum of challenges across the API lifecycle. This is where Postman Enterprise stepped in.

“Postman is an amazing platform, it provides support for the entire API lifecycle covering everything from testing to governance. It is incredibly impressive.”

Veronica Lange, Senior Software Engineering Manager, Western Governors University

WGU decided to upgrade to the Postman Enterprise plan and automate processes to align with the QE team's motto to 'automate the planet'. By partnering closely with WGU's stakeholders, Postman's Customer Success Team coordinated a variety of advanced training sessions to help WGU improve developer productivity, enhance API quality, and more.

As WGU moves towards an API-first approach with multiple teams developing and testing on a single API contract. Now, it is possible to test and build against a release during development. As a result, feature release deployment time has been reduced as changes can be monitored, and feedback is provided in real-time. Perhaps most importantly, this also means fewer meetings for everyone.

Automated Testing was incorporated into the pipeline using Postman's command-line collection runner, Newman. The Newman image is used to build modules that automate API tests in the deployment pipeline.

Now, WGU is developing against synthetic tests which are sets of API calls that pass false data to test an environment. Postman is helping drive further productivity here through collaboration: Team A develops against Team B based on API contracts using mock servers.

WGU has gained telemetry with mock servers. By deploying mock servers, a single system hits mock endpoints for all external systems. Mock servers allow continuous testing for responses resulting in efficient debugging.

Load testing now takes place in the application using a server rate of 1000 per minute to provide accurate numbers. And load, performance, and stress testing are completed using mock servers without impacting systems elsewhere such as WGU's SSO tool. WGU continues to work with Postman to increase the capabilities of mock servers to support even more.

With Postman's Enterprise user orchestration and administration capabilities, WGU has enhanced collaboration. For example, WGU leverages SSO, SCIM (System for Cross-domain Identity Management), and User Groups to organize developers into teams. These features automatically scale so that users have access to relevant and versioned API assets.

The Super Admin role, another Enterprise-edition exclusive, helps ensure that work stays organized and decluttered. Super Admins can extract and manage all of WGU's API assets from anyone's workspace.

Postman is being used for governance for the entire lifecycle of the API to ensure standards are implemented and conventions are met. WGU can now verify through security with user access control to prevent APIs from inadvertently publishing live credentials.

Developer productivity has improved significantly. Testing time has been reduced from an average of 96 hours to 5-6 per scrum team for every two-week sprint cycle.

Delivery lead time for new features and changes has been reduced to two to four weeks, which is a substantial reduction from six to nine weeks.

Automation and streamlining capabilities provided by the Postman Enterprise plan has created a fluid development cycle. Consequently, WGU's developers can now spend more time focusing on innovation and delivering amazing experiences for the faculty and students.

“Postman is very intuitive in how information is organized. Everything's in one place, and it is easy to share API cycles between teams.”

Veronica Lange, Senior Software Engineering Manager, Western Governors University

A cohesive reporting structure has been implemented, enabling WGU to gain telemetry by automating the retrieval of data from many sources, generating real-time numbers.

Testing times have been reduced to 15 minutes by utilizing both Postman and Xray. It's a reduction of 50% as before with ServiceNow it would take 30 minutes. This is significant as on average tests are run anywhere between 50 to 90 times per day, and for large multi-team deployments, tests are run in excess of 120 times per day.

For multi-team testing, the implementation of automated harnesses—modules with custom wrappers written to fit within the CI/CD pipelines—with the Newman image shortened the testing time by 75%. Multi-team team testing used to take up to four hours but has been speeded up to take only one hour.

Comprehensive testing has resulted in better quality code resulting in the business, product, and engineering teams all gaining confidence in releases. Among the many benefits of using an API-first approach is the reduction of meetings due to increased clarity of API functionality. While Postman's Enterprise orchestration and administration capabilities, along with API documentation, have eliminated duplicate work and internal disruptions.

What's next? WGU continues to work with Postman on several initiatives including increasing mock server capabilities. There are also plans to leverage Newman to incorporate automated testing for environments using a simple Postman call instead of manually triggering tests for testing collections.

In the long-term WGU is partnering with Postman's Customer Success team to implement API design standards, apply governance across its entire API lifecycle, and leverage the new features of Postman v10.