How to: Understand and explore reporting (draft)

Introduction

This “How To” will explain the essentials of understanding reporting within SurveyOptic, and how to use filtering to explore a data set, discovering insights and areas which may benefit from further analysis.

Dynamic report

The dynamic report is the primary visual reporting tool in SurveyOptic. Using the dynamic report, a data set from a survey can be viewed, filtered and further inspected using a variety of tools. The dynamic report lists the questions and sections of the survey, in the same order as set up in the survey editor. The survey title is displayed at the top of the page, with response totals shown beneath. Each question is listed with an appropriate graph displaying the results of the response set along with scales and legends to describe what is being displayed. Graph types are automatically selected based on the question type, to display data in the most appropriate format.

Common graph types and how to read them

Multichoice, multiselect, and list question types are all represented by bar graphs like the example shown below. Choices are ordered by descending frequency. Understand_explore_reporting_1

When there are more answer choices in the question than can be displayed, the lowest frequency options are rolled up into an “all other responses” field and options that weren't selected by any respondents won’t be shown at all. Questions which have been set as ordinal/ordered in the survey editor retain the order of the answer choices as set there. This is useful for questions where reports are more easily understood when the answer choices are sorted into a particular order such as this question about age. (note that even answer choices that have not received responses will be displayed in this special case)

Understand_explore_reporting_2

Likert questions use a different bar graph style, displaying sentiment across a horizontal axis. Positive sentiment is displayed on the right hand side, negative sentiment displayed on the left and neutral sentiment displayed centrally, with different coloured bars denoting sentiment of the answer choices. The bar is 'offset' to centre the neutral answers (and so if often referred to as an offset-bar-chart. Understand_explore_reporting_3

Hovering your mouse over elements within a graph will display a pop-up with more detailed information. Where the number of responses represented by a bar falls below the minimum reporting thresholds of the survey, the number of responses will be masked in order to protect individual respondent anonymity. Understand_explore_reporting_4

What to look for

When looking at a report, it is generally the things that “stick out” that are most important to take note of and perhaps explore further. We often refer to these as “interocular”, or something that “hits you right between the eyes”. The likert graphs have been specifically designed with this in mind, and are particularly effective at sticking out when they are very negative, positive or have different results to other graphs nearby. Bar charts used for multichoice, multiselect, and list question types can also be assessed for notable answers due to their high or low frequency.

Filtering

Click filters and facet filters

Filters can be used to drill down on certain groups and experiences, to discover trends within a response set. One way you can filter the responses is by clicking an answer to a question. For example, in our report I can click on the answer ‘An individual’ to filter out all the responses that did not respond as ‘An individual’. Here I can see how everyone who responded as “an individual” responded to the other questions.

Understand_explore_reporting_5

You can change the filter answer, the filter question or clear the filter in this filter tab at the top of the page. Alternatively clicking the answer again will also clear the filter. Understand_explore_reporting_6

For example, I can see how individuals who are aged 20-29 responded. With these simple filters it is easy to see that this group are generally positive about the impact the changes will have on children and the centres but feel negatively about the impact on staff. Understand_explore_reporting_8

Split filters

SurveyOptic offers further report filtering and exploration through what we call “splits”. Hovering your mouse over a question will give you the option to ‘split’ the responses by this question. This will allow you to view the responses to the other questions, separated by the answers of the question you have split by. Graphs will be flipped to the vertical axis so multiple graphs can be easily compared side by side. For example, again using our survey, if I split by; ‘in what capacity are you responding’, I can see that businesses and charities have a more positive sentiment on ‘the new arrangements have positively impacted current residents’ than individuals. Understand_explore_reporting_9

Additional tools

Any question or section within a report can be hidden by clicking on the “hide” button that appears when hovering over that element with your mouse. The [PDF report] button in the toolbar will download a PDF of the report in whatever state it is currently in. Using this functionality, targeted reports can be created focussing on specific areas. Additionally further customisation of the report is available through the advanced toolbar.

Advanced options toolbar

The advanced options toolbar can be accessed by clicking the three dots icon. Here we find a number of additional tools and display options for the report.

Enabling summary cards will show cards that summarise the overall sentiment of each section of the survey.

When question numbers are enabled, the question numbers will be displayed if there are any.

When group answers are enabled, results will be grouped within each section by answer rather than by question. This can be a useful tool when a survey uses questions with repeated answers.

When show previous is enabled the results of a previous survey will appear to the left hand side of any comparable questions in the survey being viewed.

When show incomplete is enabled, incomplete responses will be included in the graphs.

When show statistics is enabled, the number of responses to each question will be displayed as well as averages, minimums, maximums and standard deviations.

The hide sections box will hide all the section headings, the hide questions box will hide all the questions and the hide totals box will hide the response totals at the top of the page.

When show section summaries is enabled, graphs for each section will be shown.

When show instruction text is enabled… the instruction text from the survey is displayed which can help give context to the questions.

When show mean marker is enabled a mean marker is displayed on the graphs of the relevant question types. A median marker can also be enables using show median marker.

You can filter by question type by selecting the question type you want to see here. For example, filtering to see only the likert questions.

When non-offset charts is enabled, non-offset likert charts are displayed instead of the default offset graphs.

The sentiment buttons can be used to display positive, negative, neutral or net sentiment of relevant questions. The sentiment will be shown as a percentage in this column on the right.

If factors have been set-up in your survey, they will be available here to filter the report. Clicking on a factor will filter the report to only display those questions associated with the factor. In this case I can click on the “tagging” factor to see all of the open text questions that have been themed.

The arrow and lines icon can be used to change the size of the graphs displayed. Clicking this several times will cycle through the different size options.

For more information about these tools, please refer to the “Explained - Dynamic Report Page” document.

Table report, filters and cross tabs:

In the “tables” tab you will find the data of the relevant question types represented in a table format. This means, for example, I can see the exact number of responses that were an individual, a charity or a business. Understand_explore_reporting_10

You can cross-tab by hovering over a table and clicking the cross-tab button. Once you have cross-tabbed a question, a column for each answer to that question will appear on the tables for all the other questions in a similar way to when you split by a question in the dynamic report page. Understand_explore_reporting_11

Text reports:

In the “text” tab you will find the responses to the text question types with the unique responses for each question laid out in a list. The most common responses will be displayed at the top of the list. In this example the question asks about park equipment; I can see that there are 13 unique responses and that swings are the most popular choice, being picked by 12 respondents. Where reporting limits allow, these comments can also be filtered using the facet filters. Understand_explore_reporting_12