IBM case study
IBM case study
MEET MARCELO JULIANO RAMOS
Marcelo is an a Watson automation specialist based in the IBM office in Brazil.
IBM Watson is an artificial intelligence technology, based on cognitive computing that understands natural language and is capable of answering questions posed in multiple languages.
As part of his role in IBM, Marcelo works with Volkswagen on an in-house innovative new project involving Watson technology for the new Volkswagen Virtus.
This new car model will offer a cognitive manual, which utilizes IBM Watson to respond to drivers’ questions about the vehicle, including information from the car manual. The app understands natural, informal speaking and “learns” more based on the interaction with the driver. The driver can either speak, write or take photos to interact with the app. Any “how to” questions about the car can be answered by this smart assistant, simply and quickly, in multiple languages. This solution allows a new way of interacting with the vehicle and offers a new technological experience to driving.
As the Volkswagen Virtus project has many features and complex capabilities, and testing is done both in the lab and in the field for cars, the IBM and Volkswagen teams needed an organized way to manage testing and ensure the app is as quick and easy to use as the vehicle it accompanies. The app was to undergo testing by personnel from both companies, and so the teams needed to make sure they can both access and share the information collected throughout the process.
Reflecting on past experience working on an HR-focused app within IBM, Marcelo said their biggest problem was receiving enough useful feedback and managing this amount of data in a manner that allows the team to track problems, reproduce and fix them on time.
Users simply did not report on all problems because the manual process they had for reporting feedback was tedious and unfriendly, requiring a user testing the app to decide to take screenshots, attach them to an email and explain the situation.
As colleagues admitted to the team, it often put them off of reporting some issues, as the process required to much effort on their part. In addition, the received reports were not always useful, as sometimes the recordings may be partial or the information provided inaccurate or insufficient. This lead the team, on many occasions, to rework and waste time on recreating issues that didn’t have enough information provided on them.
Finally, a lot of data collected from the testing was eventually lost because there was no organized method of storing and using it in retrospect.As two teams are testing the app in its development process, they needed a way to efficiently manage testing throughout the development cycle and in transition from the IT development at IBM to a more user experience oriented approach at Volkswagen.
THE TESTFAIRY SOLUTION
After the trying experience of unorganized testing, Marcelo’s team started using TestFairy, and that made all the difference. The fact that all sessions are automatically recorded, and contain all the relevant data the development team may need to solved their biggest problem.
What actually made the testers hop on board was how incredibly easy the reporting process has become. All they had to do is shake their phone, and a bug report with a full recording and attached screenshots, logs and metrics was automatically created and sent.
For Marcelo’s team, the fact that those reports automatically open up issues on their JIRA ensured that no more bugs are missed, and all the relevant data for fixing them is available right there on that ticket. For issues users do choose to report, having all the session recordings at hand enables the team to check if other users experienced them as well.
Now, with more reports flowing in with comprehensive and accurate information, the team is able to quickly reproduce and fix problems, as well as fully understand the actual user experience, all before releasing the app.