Many of our customers use 3rd party tools to create models for hardware-in-the-loop testing. These customers want to quickly deploy those models to their hardware. Our team learned that our hardware-in-the-loop customers did not have a direct path for deploying their code to an FPGA target. As a business, we were missing out on those FPGA customers due to this critical workflow gap. We had a short amount of time to solve this technically complex problem and believed that we could benefit from using the sprint framework to test out our early ideas and set the team up for success.

TASK: Plan and execute a design sprint.

TEAM: 7 full-time participants, 3 part-time participants (deciders), and 2 subject matter experts on call

MY ROLE: Planner, co-facilitator, designer

TOOLS: The Sprint book, Pen & paper, Adobe XD, MURAL, Microsoft Teams

TIMEFRAME: A 5-day sprint in March, 2020 with research activities leading up to the start

Terminology

Hardware-in-the-loop (HIL) testing - a technique where real signals from a controller are connected to a test system that simulates reality, tricking the controller into thinking it is in the assembled product.

Field-Programmable Gate Array (FPGA) - an integrated circuit designed to be configured by a customer or a designer after manufacturing. One of the benefits of using an FPGA controller is that the user experiences almost no control latency for system response compared to other controller architectures. The downside is that FPGA code takes an incredibly long time to compile.

Bitfile - The output of an FPGA compilation. A file that can be deployed to FPGA hardware.

 The design sprint process follows the “double diamond” method. The two diamonds represent a process of exploring an issue more widely or deeply as individuals (divergent thinking) and then taking focused action together (convergent thinking).

In the weeks leading up to the sprint, our team made an effort to better understand the target user. Wanting to keep the size of the team within the recommended range, we ended up not having any HIL subject matter experts on the full-time team. To mitigate this, we reached out to subject matter experts before the sprint and had 2 additional decider/agreers attend some of our afternoon sprint sessions to weigh in on our progress and field our questions.

Day 1: Map the Problem

We started with an introduction to the sprint and then launched into our Ask the Experts session where we interviewed our SMEs. As a team, we teased out that there are two main user types involved in the HIL workflow: The Integrator and The Controls Engineer. Our team split into 2 groups and broke it down further, using empathy maps to map out these users’ daily tasks and pain points.

Next, we determined as a team that our target user for the sprint was the Integrator and we mapped out their workflow with stickies on the virtual whiteboard. We generated “how might we” statements, voted, and then our deciders came in and helped us solidify the challenge statement that would guide us for the rest of the sprint:

Day 2: Sketch

We started day 2 with lightning demos for inspiration and with our challenge statement in mind, moved onto the crazy 8s exercise. The facilitator and I made a last-minute decision to add an additional section to our MURAL board for refined drawings and silent critique. This helped us with our next exercise of affinity mapping our similar ideas.

Next, we split off into two groups to do additional exploration. After meeting up as a team to share our ideas and get feedback, we went off to do our storyboarding homework.

Day 3: Decide

Day 3 was all about deciding on a solution and determining what we wanted to learn from Friday’s testing sessions. Each team presented their updated storyboards and then did another silent vote. We decided to go with a concept that we called “The FPGA to Bitfile Connector”, a standalone product that would help the user import and convert their model to a compatible bitfile. Next, we did another mapping exercise to establish our learning goal. With that, we merged some of our concepts, created a plan, ran that plan by our handy part-time deciders, and then got cracking on building the prototype!

Day 4: Build

Keeping the team's learning objective in mind, we spent all day prototyping in a shared Adobe XD file, bouncing ideas off our main decider, and periodically checking in with the team.

At the end of the day, we tested our low-fidelity click-through prototype on an internal user as a sort of “check yourself before you wreck yourself” measure. Then we met with our UX researcher to create our concept testing script and come up with questions to ask our participants throughout the workflow.

Day 5: Test

User testing is always full of surprises. Throughout Friday's tests, we determined that users had a hard time grasping the purpose of the “prepare model” section. Since we only had 5 days to build and test, we chose to optimize for importing an NI FPGA-compatible model into our product over designing a way for users to automatically check if a model is compatible with NI hardware earlier in the process. While the participants didn't understand this first section, they didn't expect the first step to be completely automated.

We also found that there was confusion within the mapping section due to the two different user types using different terminology.

Additionally, the participants expressed that they would expect this process of going from the model to our product to be iterative, much like the HIL process itself. It is expected that any number of the export steps could fail for any reason. Our goal with this tool is to help users fix these problems. We determined through the testing that the UI should prioritize error recovery over error prevention.