How To Create A Great Bug Report

Submitting clear and accurate bug reports is the best way for you to make a real difference in the products you are testing. The product teams can address issues submitted in detailed, clearly-written bug reports more quickly and efficiently than ones submitted in reports which are unclear or lack detailed information and reproduction steps.   

• Tips for writing good bug reports
• How to submit a bug for Windows Live Products
• Example of a good bug report
• How to create a screenshot
• How to get a crash report
• What happens to a bug after you submit it?

How to keep your bug from getting sent back to you as “Not Reproduceable"

Too often Microsoft gets bugs about issues that you and others have experienced but we are unable to take action because the bug report is unclear or is missing important information. When this happens, our only recourse is to resolve the bug as “Not Reproduceable” and to request additional information. This delays our ability to fix the bug and may ultimately result in the fix getting postponed to a future release. If your bug does not follow the below guidelines there is a high likelihood that it will be sent back to you for additional information.

• Submit one problem or issue per report. One problem at a time is easier to reproduce, track, and fix.
• Attach files, screenshots and logs to every report if you can. Teams rely on this information to troubleshoot problems accurately.
• Use a clear and understandable title. Teams look at a lot of bugs. Make your bug stand out by using a clear and descriptive title.
• Create clear and detailed steps to reproduce the issue. If we can’t reproduce the issue it is almost impossible to take action on it. Check out the section of this guide titled “Tips for writing good bug reports.”
• Attach screenshots. A picture is worth a thousand words. A screenshot is often the best way to show us what’s going wrong. For help on getting screenshots check out the section of this guide titled “How to create a screenshot.”
• Include a crash log for all crash reports. This is an error that causes your computer to immediately shut down and require restarting to work.  Technicians can't find the cause of a crash without a crash report, so any bug reporting a crash that does not have a crash log will immediately be sent back. For help on getting a crash log check out the section of this guide titled "How to get a crash log."

Rating: You have the ability to rate a bug on a scale of 1 – 5. Staff will use the average ratings when prioritizing the bugs. Please use this scale when rating a bug.

1. I can live with this bug. If it is not fixed it will not lessen my satisfaction with the product and I will continue to tell my friends to use Messenger.
2. This bug is mildly annoying. If it is not fixed it will slightly lessen my overall satisfaction with the product.
3. Although this bug is not severe. I may consider another product if I don’t see that it’s being resolved.
4. This bug should be fixed. If it is not fixed I may consider alternative products.
5. If this bug is not fixed I will uninstall the product and tell my friends not to use it.

Validation: If others can reproduce this issue or if issue happened on your computer, indicate Yes, I am able to validate that this issue occurs and links to Windows Live ID's of those who also have experienced the bug or can reproduce it. If not, indicate No, I am not able to validate that this issue occurs. Bugs with more “Yes” validations will get prioritized over those that don’t have as many.

Workaround: If you have a method that can help other beta participants avoid or fix the issue enter it in this section.

Comments: If you have information that you want added to the bug or if want to view other beta participants’ comments, click on View or Add in the Comments box.

Example of a good bug report

This is an example of what a bug should look like. This is an actual bug report filed during an MSN Messenger Beta. As you can see it has clear and concise instructions that are easily reproducible for Microsoft employees.

Title: Crying Face emoticon and Disappointed emoticon appear the same in the emoticon dropdown.

Description: In the emoticon dropdown in the conversation window of MSN Messenger, the "crying face" emoticon and the "disappointed" emoticon appear to be the same. Hovering over each emoticon reveals the name; however, they are indistinguishable graphically.

System: Intel P3 900MHz 256MB RAM 60GB Hard Drive – 42GB free space Windows XP Home with SP 2 1024x768 resolution with 32bit color See system info attachment for further details

Repro Steps:

1. Sign in to MSN Messenger
2. Open a conversation window to a contact
3. Click on the Emoticon dropdown (Smiley Face)
4. Look at the first two emoticons in the second row of "My Emoticons"

Expected Result: For each emoticon to be represented by a different graphic.

Actual Result: The emoticons look exactly the same.

Attachments: < Screenshot of the Emoticon dropdown window and MSINFO32 file attached>

A screenshot is a captured image of your computer screen. Including screenshots of the problem you've encountered or the behavior you're trying to explain is an easy way to add value and clarity to your bug description. Here's how to make a screenshot:

1. Reproduce the problem or condition you want to capture in the screenshot. With the problem still on the screen, press the PRINT SCREEN key (this key is usually found to the right of the function keys and may be labeled PRT SCR on some keyboards). Windows will copy an image of your screen to memory.
2. Open up a graphic editing  program (such as Paint). Click Edit > Paste, or press CTRL+V, to paste the captured screen image into the blank file.
3. Consider using the drawing functions of your graphic editing program to circle the issue so it is super clear to us what you are reporting.
4. Save this file to your desktop. If your graphics program gives you the option to Save As a certain file type, please save the picture as a .jpg or .gif file to keep the file size down.

How to get a crash report

Windows XP and Windows Vista enable you to obtain a crash report easily by using a utility called Event Viewer.
Windows XP
To start the Event Viewer:

1. Click Start, then click Run
2. Type eventvwr, then press ENTER. You will see a log of your system events.
3. Click the Application log in the left pane to open the Application Event Log.
4. The right side of the Event Viewer dialog box window will display all application events logged on your system. Search for logs near the time of the crash. A crash should be marked with a red circle containing a white X.
5. Copy the crash information into Notepad and attach the .txt file to your bug report.

Windows Vista
To start the Event Viewer:

1. Click Start and type ‘eventvwr’ in the Start Search field
2. Click on ‘eventvwr.exe' in the search result list to open the Event Viewer
3. In the left pane, click on Windows Logs
4. In the right pane, select the application selection
5. Click on Date and Time to sort on the list and select the most recent event where Source is "Windows Error Reporting" or "Application Error"
6. The crash record is marked as ‘Error’ with a red warning sign
7. Open the record you will report on
8. Click on Copy to copy the crash information into Notepad and attach the .txt file to your bug report.

Note: it’s possible to filter on application errors using the filter button in the right most window.

What happens to a bug after you submit it?

Ever wonder what happens with the bugs after you submit them? Here is a brief description of that process:

1. Beta testers enter bugs through designated channels.
2. The Staff then makes a copy of the bug in the product team bug database.
3. The Product Team reviews all of the bugs on a regular basis.
   When they review the bugs they are looking at the following:
   
   

   Does the bug have enough information? If it doesn’t, the bug gets sent back to the beta tester with a more information request. See the section titled How To Keep Your Bug From Getting Sent Back To You As “Not Reproduceable” for additional information on the criteria.
   

   Has the bug already been reported? If the issue has already been reported during the beta it gets resolved as duplicate to the previously reported bug. The beta team will let the beta tester know to which bug it was resolved so that the tester can track the progress on the issue.

4. When all needed information has been provided by the beta tester, the team assigns a priority and severity rating to each bug. The priority and severity are used to determine if and when each issue will be fixed. Not all bugs will be fixed. The team will resolve each of those to the beta tester who opened them with a note on why they won’t be fixed.
   

This process takes time and depending on the bug it might take days or weeks to determine if there is enough information to reproduce the error, discussion might be necessary to determine if it is part of a larger error that is being tracked or any number of other variables.  Generally if more information is needed then there are requests made the first week.

I hope that those of you who like to search for bugs can use this comprehensive reporting style.  I found this recently while looking over various Beta information sheets and paraphrased it for the purposes of the participants in many of the Windows Live Betas and Windows Live Products.

Have a good one!!!

Mindy D'Amico
Microsoft MVP - Windows Live

Note: Link may not work if access is not available to reader
Excerpt from: https://connect.microsoft.com/content/content.aspx?ContentID=9640&SiteID=658&wa=wsignin1.0