Testing 2.0
Thursday, August 30, 2012
4 comments
Testing Google's New API Infrastructure
Monday, August 20, 2012
A developer’s application uses a Google-supplied API Client Library to call Google API methods. The library connects to the API Infrastructure service and sends the request. Part of the request defines the particular API and version being used by the client. This service is knowledgeable of all Google APIs, because they are defined by API Configuration files. These files are created by each API providing team. Configuration files declare API versions, methods, method parameters, and other API settings. Given an API request and information about the API, the API Infrastructure Service can translate the request to Google’s internal RPC format and pass it to the correct API Provider Service. This service then satisfies the request and passes the response back to the developer’s app via the API Infrastructure Service and API Client Library.
Now, the Fun Part
As of this writing, we have released 10 language-specific client libraries and 35 public APIs built on this infrastructure. Also, each of the libraries need to work on multiple platforms. Our test space has three dimensions: API (35), language (10), and platform (varies by lib). How are we going to test all the libraries on all the platforms against all the APIs when we only have two engineers on the team dedicated to test automation?
Step 1: Create a Comprehensive API
Each API uses different features of the infrastructure, and we want to ensure that every feature works. Rather than use the APIs to test our infrastructure, we create a Test API that uses every feature. In some cases where API configuration options are mutually exclusive, we have to create API versions that are feature-specific. Of course, each API team still needs to do basic integration testing with the infrastructure, but they can assume that the infrastructure features that their API depends on are well tested by the infrastructure team.
Step 2: Client Abstraction Layer in the Tests
We want to avoid creating library-specific tests, because this would lead to mass duplication of test logic. The obvious solution is to create a test library to be used by all tests as an abstraction layer hiding the various libraries and platforms. This allows us to define tests that don’t care about library or platform.
Step 3: Adapter Servers
When a test library makes an API call, it should be able to use any language and platform. We can solve this by setting up servers on each of our target platforms. For each target language, create a language-specific server. These servers receive requests from test clients. The servers need only translate test client requests into actual library calls and return the response to the caller. The code for these servers is quite simple to create and maintain.
Step 4: Iterate
Now, we have all the pieces in place. When we run our tests, they are configured to run over all supported languages and platforms against the test API:
Test Nirvana Achieved
We have a suite of straightforward tests that focus on infrastructure features. When the tests run, they are quick, reliable, and test all of our supported features, platforms, and libraries. When a feature is added to the API infrastructure, we only need to create one new test, update each adapter server to handle a new call type, and add the feature to the Test API.
Covering all your codebases: A conversation with a Software Engineer in Test
Saturday, August 11, 2012
Tell us about yourself and how you got to Google.
I grew up in rural Prunedale, Calif. and went to Stanford where I double-majored in philosophy and computer science. After college I spent six years as a software engineer at HP, working primarily on printer drivers. I began focusing on testing my last two years there—reading books, looking up information and prototyping test tools in my own time. By the time I left, I’d started a project for an automated test framework that most of our lab used.
I applied for a software engineering role at Google four years ago and didn’t do well in my interviews. Thankfully, a Google recruiter called last year and set me up for software engineer (SWE) interviews again. After a day of talking about testing and mocking for every design question I answered, I was told that there were opportunities for me in SWE and SET. I ended up choosing the SET role after speaking with the SET hiring manager. He said two things that convinced me. First, SETs spend as much time coding as SWEs, and I wanted a role where I could write code. Second, the SETs job is to creatively solve testing problems, which sounded more interesting to me than writing features for a product. This seemed like a really unique and appealing opportunity, so I took it!
So what exactly do SETs do?
SETs are SWEs who are really into testing. We help SWEs design and refactor their code so that it is more testable. We work with test engineers (TEs) to figure out how to automate difficult test cases. We also write harnesses, frameworks and tools to make test automation possible. SETs tend to have the best understanding of how everything plays together (production code, manual tests, automated tests, tools, etc.) and we have to make that information accessible to everyone on the team.
What project do you work on?
I work on the Google Cloud Print team. Our goal is to make it possible to print anywhere from any device. You can use Google Cloud Print to connect home and work printers to the web so that you (and anyone you share your printers with) can access them from your phone, tablet, Chromebook, PC or any other supported web-connected device.
What advice would you give to aspiring SETs?
First, for computer science majors in general: if there’s any other field about which you are passionate, at least minor in it. CS is wonderfully chameleonic in that it can be applied to anything. So if, for example, you love art history, minor in art and you can write software to help restore images of old paintings.
For aspiring SETs, challenge yourself to write tests for all of the code you write for school. If you can get an internship where you have access to a real-world code base, study how that company approaches testing their code. If it’s well-tested, see how they did it. If it’s not well-tested, think about how you would test it. I don’t (personally) know of a CS program that has even a full course based on testing, so you’ll have to teach yourself. Start by looking up buzzwords like “unit test” and “test-driven development.” Look up the different types of tests (unit, integration, component, system, etc.). Find a code coverage tool (if a free/cheap one is available for your language of choice) and see how well you’re covering your code with your tests. Write a tool that will run all of your tests every time you build your code. If all of this sounds like fun...well...we need more people like you!
If you’re interested in applying for a Software Engineer in Test position, please apply for our general Software Engineer position, then indicate in your resume objective line that you’re interested in the SET role.
Posted by Jessica Safir, University Programs
