How to become a Debug Ninja?

June 03, 2014, anysilicon

This is a guest post by Arrow Devices, that provides high-quality Design & Verification products and services for ASIC/SOC. 

Please raise your hand, if you have ever felt like putting a fork in your left eye due to debugging issues. Below is an attempt to ensure that feeling never comes back, and pave way for you to become a Debug Ninja!

So Why is Mastering Debugging Important Today?

In earlier times, debugging was something one did, but no one gave it much attention. Data was manageable, and developers didn’t pay much attention on making the process too efficient. Over the past few years, however, communication protocols have become more complex. This is true for interface protocols such as PCIE and USB as well as for bus protocols such as AXI, PLB etc. Today, RTL design and verification engineers have to sift through gigabytes of data in the form of log files and waveform dumps in order to analyze and debug a design.

By some estimates, 70% of IP design time, now-a-days, is spent on verification. A large chunk of that time (~30-40%) is spent in debugging test case failures.

The Step by Step Process to Make Debugging Easy:

So here is the good news. By following the below laid out steps, you can make your debugging process a lot more efficient.

  1. Symptom identification: Finding out what went wrong (e.g., a data integrity/protocol violation)
  2. Diagnosis: Analysis of what went wrong for symptom to occur
  3. Scenario Isolation OR hypothesis formulation: Identifying the exact sequence of events that is unique to issue under consideration
  4. Scenario conformation OR hypothesis testing: Cross checking that the identified scenario or hypothesis formulated actually resulted in the issue

Let us go through these steps more carefully, one by one:


Step 1: Symptom Identification

The first step is for the engineer to figure out what went wrong, by looking at high level symptoms. For example, did a data integrity error occur? Or was it a protocol violation? In order to be most efficient, it is important to identify correctly, the broad area of the issue first, and then systematically probe further, once the broad issue is correctly identified. Not doing this, would lead to a possible wild goose chase.


Step 2: Diagnosis

Next, the engineer needs to perform diagnostics by analyzing what went wrong for the symptom to occur. For example, did the data integrity error occur due to a data packet being dropped?


Step 3: Scenario Isolation or Hypothesis formulation

Once the engineer has correctly diagnosed the problem, he needs to isolate the scenario under which the problem occurs. If its not clear under what all scenarios the problem occurs, he needs to formulate a hypothesis about what kind of scenarios are problematic. For example, is it possible that a data packet is dropped whenever there are 2 back-to-back data packets with the same size?


Step 4: Scenario Confirmation

The engineer now needs to confirm that the scenario he has identified is indeed the problematic case. This can be done by analyzing all the errors seen and checking that the scenario hypothesis holds true. Once the scenario is confirmed, the engineer can rest assured that his fix will correctly address the bug or issue seen.

Following these steps, not only makes the debugging process more efficient, it also leads to standardization of the process across a team. As a manager, this is important, as he or she can help the team more meaningfully, if they know what stage of debugging process has been reached. It also helps in improving debugging quality.


So is This All I Need to Know?

No. Even if you have mastered the above steps, and follow them religiously, there is the problem of sifting through gigabytes of data that we mentioned earlier. A developer needs to extract relevant debug information from huge wave form dumps or log files. This is a problem. The current representations of data are not friendly for the drill down steps described above. In order to go through these steps quickly, the design engineer would need to view the data top-down. This means, the data should be available at multiple abstractions and one should be able to search, filter and mark information quickly -at the click of a button. At Arrow Devices, we saw this as an problem worth solving, and so we went ahead and developed our debugger tool – The Protocol Debug Assistant (PDA). Check out the details of this tool, on our website link here


The best way to figure out if you need a debugging tool, is to ask the following questions:


1. Does the tool help me drill down to issues top down? In other words, does it give me the option to look at data at various abstractions?

2. Does it allow me to filter and mark events, that I want to drill down to?

3. Can it be used with my current verification environment?

4. Do I need to customize it to the particular protocol I am working on?

5. Does it help standardize the debugging process across the team and share the debug knowledge?

6. Finally, does it help me reduce the time I spend on debugging? Or is it an overhead?


To read more from Arrow Devices, please visit their blog.