Reputation builds a custom test automation framework on Postman

Reputation's engineering team had been using the free tier of Postman for a number of years. Primarily, developers and QA engineers would do basic sanity testing on Postman of internal APIs that were exposed by the platform's microservices.The testing was completely manual and the team would export and share their Postman Collections through email or Slack. This was good for testing core functionality but not 100% reliable for catching regressions and running it frequently (like after every PR merge). In addition, the import and export of collections would increase the risk of code conflicts due to Postman's free tier collaboration limits.

“We needed a more robust automation framework which could significantly increase our end-to-end coverage, be easy to write, and keep pace with our feature roll outs.”

Shanmukha Santosh Kumar Kunapareddy, Lead Automation Engineer, Reputation

Reputation evaluated other solutions before deciding to upgrade to Postman Enterprise due to engineers' familiarity with the platform, Postman's ease of onboarding and use, and the large number of Postman Collections already established across the organization helped in this decision.

“Getting started with Postman is dead simple with almost zero learning curve. Adding tests is a breeze in Postman, and this helped in promoting it as a tool for automation.”

Shanmukha Santosh Kumar Kunapareddy, Lead Automation Engineer, Reputation

Once upgraded, Reputation began by configuring Postman Workspaces for every team, adding members and collections, and syncing everything between Postman's cloud and desktop clients. In particular, storing collections in the cloud made it much easier for users to share, update, and collaborate. Testing was also easier—the organization only needed a brief training session on how to use the additional “Test” tab in the Postman interface, and how to write test cases and validate responses.

“It was quite easy to repurpose the plethora of collections lying around in developer and tester laptops, add assertions, and voila! Our test suites were churning out fast.”

Shanmukha Santosh Kumar Kunapareddy, Lead Automation Engineer, Reputation

From Ad Hoc to Automated Testing

QA engineers developed a common framework for automating API testing on Postman. A framework was built using Node.js which exposed gulp tasks to run the collections, setup and teardown the database after test execution. The framework also integrates Postman with GitHub to help streamline the team's CI/CD pipeline.

To execute tests, a Gulp task runner invokes Newman to run collections directly from the command line, with Postman seeding all the setup data in JSON files. Now, whenever a developer forks a collection in their workspace, they add the necessary tests and raise a PR in Postman. The PR gets merged and code is committed to GitHub. Every two hours, Jenkins pulls the latest test code from GitHub and executes it against the test environment in Docker. The Newman plugin for reporting generates reports that are then mailed to users after every test run.

Recently, Reputation collaborated with the Postman team to integrate Jenkins directly into the Postman API Platform, which allows the team to view the status of builds or kick off a new build all from within Postman.

“When we had a need for new functionality, Postman listened to us. We were able to collaborate on a new feature that will improve our test automation pipeline even further.”

Shanmukha Santosh Kumar Kunapareddy, Lead Automation Engineer, Reputation

Currently, about one third of the Reputation's 160+ engineers are using Postman Enterprise along with the QA team's test automation framework, but interest has been growing. As with the company's core business, user feedback and word of mouth are key for internal tools as well, and Postman's reputation continues to rise across the organization.

To learn more about Reputation's testing framework, read Kunapareddy's blog post.