When designing a product, delivering something that offers a viable solution to the audience is the final step of a long and iterative process. As you try to do this while creating something that can establish itself in the target market as well, failure at any stage is just a part of this long and complicated process. This is where prototype testing comes in!
Before a product goes out for public access, the developers need to ensure that it is fully operational, offers promised functionality, and solves real-world problems. Testing prototypes is an ongoing process where the developers seek continuous validation from real-world users through feedback and surveys.
That’s the reason why developers hold prototype testing in high regard. It is the single most effective way of knowing how a product will perform in the market before it performs.
By now, you have a basic idea of how important prototype testing and evaluation are and why developers consider it one of the key elements of the development phase. You may still be wondering what prototype testing is, so let’s take a closer look.
What is Prototype Testing?
Prototypes are the first draft in the product development phase before we reach the final product. When these prototypes are circulated to real-world users to find flaws and areas of improvement, it is known as prototype testing.
You can deploy a prototype in the field for people to use your product before finally launching it to ensure that you offer the best quality. A process known as crowd-testing is one of the most effective ways of analyzing the quality, features, and other crucial components.
Crowdtesting offers the prototype of the environment that it would have to function in the future. Developers can use such conditions to assess how the product behaves under different circumstances.
Knowing how a product would behave in the market is a crucial component of prototype testing. Still, developers always keep note of how the users would interact and perceive the product is equally important.
Different Types of Prototypes
Prototypes offer the basic functionality of product discovery. Each prototype has an ideal set of conditions, which it delivers the cheapest and fastest way of testing your assumptions.
There are 4 different types of prototypes, and they are as follows:
1. Low-Fidelity Prototypes
These types of prototypes are mostly paper-based and also do not allow any user interaction. Any hand-drawn sketch or a mockup on a sheet of paper is a low-fidelity prototype.
2. High-Fidelity Prototypes
On the other hand, high-fidelity prototypes are highly interactive and functional, and they are as close as they can get to the final product. These prototypes contain all the necessary components integrated into them.
3. Live Data Prototypes
These prototypes are mostly in the form of codes, often on top of the existing products. They are much more interactive than traditional prototypes and use live user data.
4. Feasibility Prototype
This is a more specific kind of prototype that is explicitly designed to test a specific feature or component of the product. These prototypes function within their pre-defined boundaries to validate the technology behind a specific function.
Now, let’s look at what these prototypes can test and when is the perfect time to put them to the test.
Why Do You Need Prototype Testing?
Prototype testing is not just about gathering feedback and insights from the users and giving that information to the developers. There are a lot of benefits that prototype testing and evaluation offer; let’s have a look at 4 top ones:
1. Reduces Overall Time and Cost
Fixing a product while still in the testing phase is a much simpler and easier way of proceeding with the launch. Because once you launch the product, you’ll need a lot of time and money to roll back the launch and the changes.
That can damage not only your finances but also your reputation – which is invaluable. That’s why prototype testing gives developers the perfect opportunity to fine-tune their product before it is released.
2. Increases User Involvement
When you begin with the prototype testing stage, users get a first-hand exclusive look at the product before anyone else in the market. This is a great opportunity for the developers and the organization to get users involved with the product. It ensures user’s commitment towards the product, and they can also help in spreading the word.
3. Gathers Accurate Requirements
Traditionally, requirements are gathered before a product goes into the development stage, and this is where people decide what they would want their product to deliver.
Conceptualizing the product without seeing it and establishing requirements is not exactly feasible, which is why prototype development allows users and developers to solidify requirements for the final design.
4. Helps Resolve Conflict
When a team of highly skilled and trained developers is working on a product, they may encounter several instances where some believe that a certain implementation is a right thing to do. In contrast, others may think otherwise. Inevitably, differences in opinions may exist during the development stage, giving rise to conflicts.
By testing the prototypes, developers can conduct several different feature iterations and benchmark the resulting performance. This way, they can establish which function generates the best response from the users based on numbers.
There are many other upsides to prototype testing, but knowing why you should test your prototype is simply not enough. When to test your prototype is also another important consideration.
When Should You Test Your Prototype?
Putting your prototype to the test is the safest way to proceed forward to ensure that your product does not meet its inevitable end. But this has some complications because knowing when to test your prototype is also essential and the only way to gather results that carry significance.
We have compiled a table that will give you a better understanding of the prototypes and when to test them for maximum significance.
|Prototype||Stimuli||Testing Ability||Testing Time|
|Layout and designs
design ideas and experiments
|Right after completing the design work and early development stages.|
|Overall design direction
|After making final changes once you receive low-fidelity and user test data.|
|Live data prototype||API
|Overall functionality||When the developers need proof if something works.|
|Feasibility prototype||Software codes
|Fundamental feasibility||Before committing resources and while defining the scope of the project.|
By now, you should be very well aware of the different types of prototypes and when developers should test them for maximum efficiency. It’s time to look at the steps that you should follow to test your prototype.
How to Test Your Prototype: Step By Step
Testing a prototype is a complex and sequential process. But before we get into the steps, there are a few basic rules that every prototype tester should glance through:
- The type of prototype you have (sketch, physical prototype, paper interface, digital prototype) will define how your users interact with it.
- Your testing goals are extremely important, and they can help you develop testing scenarios, questionnaires, and the overall prototype testing method.
- Setting time constraints will also help you decide the best prototype testing method for you.
Here are the steps that need to be followed with precision to ensure that your testing yields the most effective results:
We’ll take you through all of the steps in detail, so follow through:
Step 1- Collect and Analyze User Information
The first step of any prototype testing and evaluation is collecting and analyzing the user data and information. Here, the users or the general public are liable for giving their verdict on what they expect from a particular product.
Once the users have given their verdict and all the requirements are in place, you can take further steps. Clarity is what we look for in the first step, and knowing what to do is the only way of avoiding being vague in the development phase.
For instance: Qualaroo specializes in collecting user insights quickly and effortlessly at every stage of the design process. The user insights are collected by staging prototype URLs via InVision, AdobeXD, and many more prototype testing tools.
These prototype testing tools host the URL mockup and allow users to test and give their feedback on the overall experience. The development team takes up this feedback and insights, making necessary changes in the final product.
Related Read- Looking for a Feedback tool? Here is a list of Best Customer Feedback Tools For Users
Therefore, the requirements are defined beforehand, and everything in the prototype is built around those requirements. To give you a better understanding of clarity of requirements, here is a quick example of prototype testing:
Being vague and unclear while designing a prototype
- Is my design good?
- Do users like the product offering?
- Is the product creating engagement?
Clarity of steps while designing a prototype
- I would like to know if users can easily interact with the design and navigate their way through it.
- I would like to know if the users would be able to perform a certain task.
- Does the prototype influence users to engage further and make a purchase?
Step 2- Build The Prototype
The most important yet most obvious step is to build a prototype of the product that is to be tested. The type of prototype you would be building depends entirely on what you need to test and at what stage of product development you are currently at.
The prototype built in this stage is derived entirely from the information that the developers gather in the previous stage while building a preliminary prototype. This prototype is supposed to be a more refined version of the preliminary prototype that will go into the prototype testing software and should ideally give more information to the users about the product.
In case you don’t remember the types of prototypes that we discussed earlier, here is a quick look at 4 different types:
- Low-Fidelity prototypes
- High-Fidelity prototypes
- Live data prototypes
- Feasibility prototypes
Things to keep in mind:
- Create digital prototypes that resemble the product – Once you’ve created a preliminary design, you can create an interactive and digital high-fidelity prototype that resembles the final product closely. This would allow more testing and make it easier for the users to understand.
With digital prototypes, you can also automate the process of collecting insights using tools like Qualaroo that enable you to ask users questions during testing or after.
- Use real data – While building prototypes, it is also advisable to use real data for the prototypes. The prototype need not be a finalized copy of the final product, but it should contain real data to accurately test your microcopy through the prototype.
- Test 2 to 3 prototypes – Don’t put too much time and effort into what’s “only” the prototype of that solution. It’s more useful to have 2-3 rough prototypes to test than have 1 pixel-perfect prototype. This lets you test more designs and also test participants are not restricted to one design that may seem like the ‘only’ solution.
Step 3- Decide What You Want to Test
This step primarily focuses on you, the tester. You should find out what you want to test before you put your mockup through prototype testing software.
There are several things that you can test on prototypes with some obvious imperfections. Here is a list of those things:
1. Concept validation- These tests are relatively simple, and they verify if the users can easily understand what they are looking at and what function it performs. These tests are most commonly used for home page prototypes but you can also test product pages in e-com or dashboards in online tools.
2. Navigation- Navigation is another thing that you can easily test on digital prototypes. Here, a mockup of the website or the application is hosted, and testers see if the users can easily navigate their way through and find what they are looking for. You can get answers to questions like:
- Are your search field and menu where people expect to find them?
- Does their naming make sense?
- Can people find what they think they’re going to find based on the categories you have presented?
3. Design flow and functionality- Prototypes also effectively determine if the products and the functions have a smooth flow and can let users accomplish the task instead of confusing them.
4. Microcopy- To test the microcopy, you need to input real labels, menu categories, buttons, and descriptions in your prototype. It will verify if the users can understand what they are looking at or confusing bits and pieces.
Knowing what you shouldn’t expect to test on your prototype is important as well. Here are a few aspects to keep in mind:
- Graphic design – Prototypes are just schematic representations of your final product, and they may not have all the visual elements. That’s why testing “look & feel” on prototypes is not possible in most cases.
- Content – Prototypes are not filled with the final content, and are not the best way to check if your content will resonate with your target audience. To check this, send your content to a couple of people from your target group or display questions in your working product or blog.
- Volume testing – The idea behind testing the prototypes is to collect quality feedback on the functional design so that you can iterate and eliminate the largest issues that can hinder your user’s ability to complete their tasks in your product.
Prototypes are not intended to help you collect volumes of data. Technically it’s possible, but insights from a couple of participants will deliver enough insights to make the next version better. Prototype testing is about gathering actionable feedback fast, not collecting as much feedback as you can.
Step 4- Create a Preliminary Design
A preliminary design is a simplified form of the final prototype that gives the users a rough idea about how the final prototype would look.
Benefits of preliminary design:
- Provides an accurate blueprint for the final prototype.
- Ensures mutual understanding about the design between the users and the developers.
- Prevents further mistakes by rectifying them in the preliminary design stage.
- It also identifies risks that the design could face in the near future by comparing it to identical projects and prototypes.
- provides a reference point and opens up a dialogue between the developers and the users to share opinions and views concerning the design. If the design does not seem fit to the users, you can easily implement changes, and create a new preliminary design.
To create it, keep your final design’s basic plans and specifications in mind before going to the drawing board.
Start by creating rough sketches. Sketches are a great way to identify if your users can discern the purpose of your application or website. Sketches and paper prototypes do not require perfection. Instead, you can have multiple prototypes to test different designs and for more meaningful insights.
Step 5 -Prepare Your Test Scenarios and Questions
Usability testing is about giving users a specific task to perform that’s linked to the problem your product or website is aiming to solve. Tasks (or scenarios) have a form of small narratives. They’re typically brief but still give your test participants some context.
This is the moment when you use your research questions to compose your tasks. Your research questions will tell you what the tasks should be about.
- Focus on users’ goals, not on your product’s functionalities and features
The best example illustrating the difference is a usability study conducted by Jared Spool and his team for Ikea, years ago. The test explored how people found products on Ikea’s website.
The initial task was: “Find a bookcase”, later it was changed to: “You have 200+ books in your fiction collection, currently in boxes strewn around your living room. Find a way to organize them.”
The way the task is formulated influences the results. In this case, users following the first task usually typed “bookcase” in the search field. Users in the second scenario were usually browsing through product categories and searching for any products that would be suitable for storing books, not necessarily products named “bookcase”.
In the end, the problem a user tries to solve here is finding furniture to put their books on, whether or not that’s a bookcase.
This is particularly important when designing products or websites that use very specific language. Try to avoid words that would be leading and make your users accomplish the task faster or in a different way than they would normally. Try to not give clues in general.
- Make the task resemble real life
If you are asking your user to book a flight via your app/ prototype don’t just say “Book a flight from Seattle to Amsterdam”.
A better scenario would be: “You want to visit your friend in Amsterdam in September. You booked two weeks off at work. You realize it is an expensive flight but you would like to spend as little as possible. In addition to this, due to your recent back problems, you are considering upgrading your flight class.”
- Do not explain your product or website or where you sell it
The whole idea of testing is to verify whether or not people will be able to use it on their own, without anyone explaining anything to them prior and without anyone persuading them they should be using it.
What’s more, especially if you are doing a face-to-face test, users may be reluctant to be honest with any criticism of the prototype if they see you are so attached to it – they will not want to hurt your feelings.
- Do not develop tasks that are impossible to complete
If your prototype doesn’t include the feature, flow, or elements you want to test, you cannot test it.
Forming Research Questions
Research questions are questions you are trying to find answers to by asking users to carry out different scenarios with your prototype. Research questions indicate what exactly you are trying to find out about your prototype or product.
Tips to form research questions
- They should be composed carefully as they will set the direction of your test and determine what the scenarios and tasks for the test will look like.
- They should not be too general. Keep in mind that based on the outcome of the test you will want to make some design decisions.
- Research questions can also be formed as goals.
Example of a bad research question/goal:
- I want to test my prototype.
Example of a better research question/goal:
- I want to test my navigation.
Examples of good research questions/ goals:
- I want to check if users will be able to find the information they’re looking for in my prototype.
- I want to check if users will be able to find the product they are looking for in the prototype.
- I want to check which version of the prototype seems easier for users when it comes to finding a specific product on it.
Also, some of the appropriate prototype testing questions to pose in front of the audience would include:
First Glance Testing
- Who do you think this tool/website is for?
- What are your initial thoughts about this application/website?
- What do you think you can do using this tool/website?
- When would you use it?
- Who do you think this is for?
- Is there anything this tool resembles? If yes, what?
- Is there anything here that doesn’t make sense?/Does anything feel out of place?
- Was there a particular function that was missing?
- How would you rate the difficulty level of this task?
- Was anything out of place? If so, what was it?
- How did you recognize where the products were?
- Does paying via this tool/website feel safe?
- What information about the products was missing?
- Do you think the tool/website guided you to the product perfectly?
- Was the application easy to navigate through?
- How was the overall design experience?
- What would you change in this application/website, if anything?
- How would you improve this tool/website?
- Did the tool/website perform the way you expected?
- How would you like to rate your overall experience?
Related Read – Here is our list of 30 best A/B Testing Tools
Tip: We suggest not using more than 4-5 questions in a sequence (after each task) since this can disturb the flow of the test and can cause fatigue.
- If the test is face-to-face you will probably be asking additional questions that will pop up during observation to probe the difficulties participants encountered.
- If this is a remote test with a tool like Qualaroo, more than 4-5 questions will result in a lower response rate, and since you’ll be only testing your prototypes on a couple of users, the response rate is important.
Depending on how much time you have for testing and what the scope is, you should have 1-5 research questions developed. This doesn’t mean you cannot observe other aspects of your design being tested.
In fact, every time you conduct user research, there almost always will be plenty of other learnings apart from what you were directly testing. You should still always have 1-5 core aspects you want to test/analyze.
Step 6- Initiate User Evaluation
To begin with, you need to conduct an initial user evaluation, where the proposed prototype design is presented to the users. All the comments and suggestions from users are considered in this step alone, and developers work upon those comments and suggestions in further steps.
The primary aim of initial user evaluation is to identify the strengths and weaknesses of the prototype design. The only way to effectively do it is by having multiple people give their unique perspectives by responding to the prototype testing questions.
Other factors to consider are that you should always present your prototype to the right audience and always ask the right questions.
The audience should consist of users who would be using the product when it’s launched, and the questions should directly be related to your product at all times. This is the only way of ensuring that the suggestions you get are meaningful and would add to the value of the final product.
For instance, if you are launching a hotel booking application/website, presenting the prototype design only to people who travel frequently and stay in hotels would make sense.
Related Read- Here is our list of 11 Best Mobile In-App Feedback Tools
What to Do After Prototype Testing
After prototype testing, you have to start refining and making final changes to the prototype. Before you launch, monitoring is also necessary to ensure that everything is in place.
1. Refining and Tweaking Prototype
This step can only begin once the developers have gathered and assessed the data from user testing.
The developers critically examine the feedback and user data, and this step continues until every last change has been made to the prototype as specified by the users. After making the changes, the prototype is again given to the user for testing and gathering feedback.
This loop continues until the users are completely satisfied with the prototype and no further changes are needed.
2. Implementation and Monitoring
The final step is to launch the final product in the market. Before launching the final product, you can always choose to pilot test it and see for yourself that everything is in order.
To pilot test your product: you need to use a prototype testing tool and select a group of end-users who would try the product for themselves and provide final feedback before the full-scale launch of the product.
You need to make sure that the end-users should consist of people using your product once it’s launched.
This is always a recommended way of scientifically implementing something as it offers maximum surety to the developers. If anything is missing from the final product, it can always be corrected before the product launch.
Once the developers have finalized the prototype after thoroughly pilot testing it, the team can launch the final product in the market. A team of developers should be responsible for continually monitoring the performance of the new product.
After the implementation, monitoring is extremely important because it will ensure that the product does not fail and can cater to all the users in the desired fashion.
Before you go and test a prototype for yourself, there are some important tips that you should know about prototype testing. These tips will come in handy when you are testing, and they can guarantee the best results.
Best Practices For Prototype Testing
You can conduct prototype testing in several ways, but we have compiled a list of best practices that you should always keep in mind while conducting your prototype tests.
1. Prioritize Usability
One of the most important bits of prototype testing is to ensure that it can easily be used by all the audience members in the real world. The product’s usability can only be maximized by gathering unbiased opinions during the new product testing phase.
To do that, you should always reach out to the general public who have no prior information about the product. By having a fresh set of eyes on the product, you will be able to assess the product from a completely different perspective, giving you an honest look at how your product is to be used by the general public.
A popular way of collecting user feedback is through emails and feedback forms. However, using surveys embedded in the product is much more effective and will provide contextual feedback. Using Qualaroo for embedding surveys in your product will allow you to gain 10 times more valuable insights than email surveys.
2. Do Not Aim for Perfection
Prototypes are not designed to be perfect; they are designed to be insightful and informative. When developing your prototype, the only thing you need to keep in mind is not to make these perfect but to make them in a way that they can convey information to the public and gather feedback at the same time.
Prototypes should be designed to help testers and developers improve these by identifying their strengths and weaknesses through feedback and insights.
Remember you don’t have to fix everything.
Try to figure out which usability problems are critical for the user. If you cannot make a decision, invite a person or a few people to a good old debriefing session. Share the results you have with them and try to pick their brain. You don’t have to invite people from outside of your company, and the people you invite do not have to be UX experts.
Be realistic about how much you can fix before the next round of testing or before handing in your designs to the dev team.
3. Keep an Open Communication
It is always helpful when your users can communicate to you about how they feel about your product. Keeping an open communication would allow the developers to get accurate and up-to-date information through user experience, which you can use to make necessary changes during the new product testing for a better user experience.
4. Develop Buyer Personas
When designing a product, you should always be mindful of the audience using your product. Keeping in mind the demographics of your target audience will allow you to design your product in a way that is preferred by your audience.
Developing accurate buyer personas also proves to be important at this stage. By doing so, you will be able to help your team develop a shared understanding of the targeted buyers and predict their behavioral patterns accurately.
Buyer personas will help you make decisions strategically instead of intuitively.
Related Read- To learn more about building customer personas, take a look at How to Build Customer Personas: The Complete Guide.
5. Be Considerate Towards Your Audience
If you are collecting any personal information about your test participant, get their consent first. This also applies when you are recording them while they test your prototype. You should be particularly careful when conducting the study in the European Union, where GDPR would apply.
However, please mind that outside the EU more and more countries (and some states in the USA) are introducing similar regulations.
Make sure you not only get the consent to collect information on participants and/or record them but also inform them these will only serve to prepare conclusions and a summary of the study and that they will only be used internally and not published anywhere.
6. Give Clarity to the Participants
Make it very clear that the usability test (or UX/ user/ prototype test/ study/ research) is not about testing the user (them) but testing the functional design. This means participants can only help us verify whether the prototype is good or not. Be sure to tell your participants they can’t be wrong.
7. Tell Your Participants You Did Not Build the Prototype
If they think you are the one who developed the prototype they will refrain from critical remarks to not hurt your feelings. To encourage honest feedback, be open and engaged. If this is a face-to-face study, don’t defend the prototype and design solutions in it. Be neutral and try to avoid emotionally loaded words whenever you are describing the prototype or its elements.
8. Conduct Tests in a Wide Spectrum of Environments
After designing your prototype, your testing should be as diverse as possible. You should ensure that your new product testing is done by a diverse group of audiences in a range of different environments. This will allow you to know which environments work in favor of your product and which ones don’t.
It is very much possible for a product to succeed in one environment and completely fail in another. But the only way of knowing this is to actually put it to the test in a wide spectrum of environments.
If you are launching a hotel booking application, then your testing environments should typically involve users from different geographies and of different age groups who travel frequently. You can segment your customers based on different demographics like age: young (18-25), mature (26-35) and adult (36 and above), gender, income, and also different states of the country.
This will paint an accurate picture of people representing different demographics and geography, based on their response and likeliness, to opt for your hotel booking application instead of conventional modes of booking.
Related Read – To learn more about recruiting participants for your research, take a look at Recruiting User Research Participants with Qualaroo
1. Begin Testing as Soon as Possible
If you keep waiting for your product to be completely developed, you’re missing some crucial insights. It’s obviously not possible to start testing right after coming up with the product idea. At the same time, you shouldn’t wait until your product nears the final stages.
You should start testing your prototype right after creating the preliminary design.
Early testing won’t be able to cover all the details of your prototype, but it will surely single out problems with the preliminary design. Early testing would also allow you to start working on the problems right away, and your designs can be improved from the get-go.
2. Map Out All Your Tasks
All your testing sessions should have a clearly defined goal. Even though the primary aim of the testing process is to get valuable insights, you should have an actionable plan in place to make your process more efficient and seamless.
Aim to create targets and get answers about specific tasks and experiences through actionable steps that are easy to track.
Instead of being vague about your objectives and tasks, be specific about what you want the information you need, and create a blueprint and prototype testing template beforehand.
3. To Moderate or Not to Moderate
Moderated tests involve a person facilitating the test through a moderator, and unmoderated tests do not have any such moderators.
Now you may be wondering whether your tests should be moderated or not.
- These allow you to have more control over the test
- You can let participants skip or focus on specific areas
- Especially recommended if the testing process is long and has potentially complicated bits that users may find difficult to understand.
- These do not allow testers to follow up or intervene in the testing process.
- Unmoderated tests are recommended when you want completely unbiased feedback and when your tests are relatively simple and easy to understand.
- Unmoderated tests will allow you to gain a lot of insights while saving crucial time.
- These are more scalable and also helpful with remote tests with users in different time zones.
4. Test Your Test
You may want to pilot test your test, especially if this is a new domain for you. A trial test will help you quickly pick up anything you forgot about and will help with understanding if the participants will easily pick up the tasks you prepared for them.
This doesn’t require too much preparation or recruiting anyone specific. Just ask anyone from your company to sit in front of the screen for a moment and read the tasks. If something is not clear you will know immediately because they will be confused and will start asking questions right at the beginning.
This is very useful especially when you are trying hard to choose the right words to describe the tasks and avoid leading or emotionally loaded words because sometimes you end up with a version that is too tricky to be understood by the average person.
Test Your Prototypes With Confidence
Prototype testing may not be the simplest thing to pursue and can take you a few attempts to come around. But it will always open new doors and pose new prototype testing questions. Therefore, it is safe to say that your prototype testing should reveal:
- What worked
- What did not work
- New questions
- New ideas
- Overall evaluation in general
At the end of the day, if your product solves the problem you initially set out to solve, you can safely consider it successful prototype testing and product development.
A successful prototype testing requires a good feedback tool, and this is where a feedback tool like Qualaroo can help your development team get actionable insights and uncover what your audience thinks.
By embedding Qualaroo in the product through NudgesTM, you get valuable feedback directly from your audience that can help you make significant improvements to your prototype’s functionality.
Watch: How to use Nudge for Prototypes
Want insights that improve experience & conversions?
Capture customer feedback to improve customer experience & grow conversions.