Success Metrics for a Data Help Desk

With in-person events, the number of interactions with researchers can be a key metric to understand the reach of the event, along with the qualitative metrics of how engaged those individuals were (did they ask questions and did they feel like they received the help they were looking for?).

Success Metrics

There are a number of different metrics that can be tracked using simple and fairly cheap applications, including paper.

  1. Impressions
    • Number of times twitter users have seen posts containing the tracked hashtag (one user can make multiple impressions and this may only mean that it showed up in their Twitter feed)
  2. Posts
    • Number of posts that used the hashtag
  3. Reach
    • Number of unique users who have seen a post containing the tracked hashtag (posts with this hashtag showed up in the feed of this many people)
  4. Engagement
    • Number of intentional interactions (retweets and likes) with a post containing the hashtag
  5. Users
    • Number of users who posted with the hashtag

While the metrics above can demonstrate that the event had a broad reach, none of them capture how engaged individuals are with the resources they see - this is something that could be at least qualitatively assessed at an in-person event. In addition, the organizers of past Data Help Desks do not weigh heavily the number of questions asked at the events because we believe that it will take time for researchers to understand what questions the Data Help Desk can help them address and it may take them time to realize what questions they should have. The value of raising awareness of FAIR concepts and techniques cannot be overlooked and can be of great benefit to funders, publishers, journal editors, data facility professionals, and others looking for a mechanism to share leading practices.

One final note on metrics: you will want to track metrics for the volunteer experts to try to assess their overall satisfaction with the activity and to find ways to maximize the return on their investment of time. The number of volunteers can be telling, as can a post-event survey shared with them. Tracking engagement related to their contributed resources can also demonstrate value to the volunteers.

Measuring Success

The success of a data help desk can be measured through a variety of quantitative and qualitative methods. Here are some key ways to determine if a data help desk has been successful:

  1. Number of interactions
    • The quantity of people interacting with the help desk can be tracked and is a direct measure of its reach and visibility.
  2. Question and answer quality
    • Assessing the quality of the questions asked and the answers provided is crucial. This might involve anecdotal feedback from users and volunteers or a more formal review process.
  3. Recording questions and answers
    • Keeping a record of the questions asked and the answers given allows organizers to identify recurring themes, knowledge gaps in the community, and areas where more resources are needed. This record can also inform future training and content development.
  4. Surveys and questionnaires
    • Utilizing surveys and questionnaires at different stages (before, during, and after the help desk) can provide valuable insights.
    • User surveys can gauge satisfaction with the help received, identify unmet needs, and gather feedback on the effectiveness of the service. High levels of satisfaction reported by users indicate success.
    • Volunteer surveys can assess their experience, identify areas for improvement in the organization and support provided to them, and determine their willingness to participate in future events.
  5. Qualitative engagement (in-person)
    • For physical help desks, observing the level of engagement of individuals can be informative. Did they ask follow-up questions? Did they seem satisfied with the information they received?
  6. Metrics for online interaction
    • Several metrics can be tracked for online platforms:
      • Impressions: The number of times posts containing the tracked hashtag were seen by users.
      • Posts: The total number of posts using the hashtag.
      • Reach: The number of unique users who saw posts with the hashtag.
      • Engagement: The number of intentional interactions like retweets and likes.
      • Users: The number of individual users who posted with the hashtag. These metrics can demonstrate the broad reach of the event.
  7. Volunteer satisfaction
    • Ensuring that volunteers feel their time was well spent and are willing to participate in the future is a key indicator of a successful and sustainable data help desk. Tracking engagement with their contributed resources can also demonstrate value.
  8. Understanding community needs
    • The types of questions asked can highlight the community’s areas of need and inform future efforts in data management training and resource development. Analyzing the questions can reveal recurring challenges, such as acquiring data, data cleaning, archiving, and understanding FAIR principles.
  9. Long-term impact
    • While harder to measure immediately, the long-term success could be reflected in improved data management practices within the community and an increased understanding and adoption of FAIR principles.

It’s important to note that organizers do not always weigh heavily the number of questions asked, especially in the initial stages, as it may take time for researchers to understand the help desk’s purpose and the types of questions they can ask. Raising awareness of key concepts like FAIR data can also be considered a success in itself.

Determining the success of a help desk

The success involves evaluating various metrics that reflect its performance in supporting users, resolving issues, and providing excellent customer service. Here are some key indicators to assess the success of a help desk:

Number of Interactions

  • Ticket volume: The number of tickets submitted through the help desk system.
  • User engagement: The number of users who interact with the help desk, including phone calls, emails, chats, and in-person visits.

Question and Answer Quality

  • Response time: The average time taken to respond to user inquiries.
  • Resolution rate: The percentage of tickets resolved within a certain timeframe (e.g., 80% of tickets resolved within 24 hours).
  • First-call resolution (FCR): The percentage of tickets resolved on the first call or interaction.
  • Escalation rate: The number of tickets escalated to senior technicians or management due to complexity or severity.

Surveys and Feedback

  1. Customer satisfaction surveys: Regular surveys sent to users to gauge their satisfaction with the help desk’s services, including response time, resolution quality, and overall experience.
  2. Net Promoter Score (NPS): A measure of user loyalty and satisfaction, calculated by subtracting the percentage of detractors from the percentage of promoters.
  3. Feedback forms: Users’ feedback on specific issues or interactions, providing insights into areas for improvement.

Performance Metrics

  1. Average handle time (AHT): The average time spent handling a single ticket or user interaction.
  2. Ticket resolution rate: The percentage of tickets resolved within a certain timeframe (e.g., 95% of tickets resolved within 24 hours).
  3. Abandonment rate: The percentage of tickets abandoned by users before resolution.
  4. Repeat business rate: The percentage of users who return for repeat support or services.

Key Performance Indicators (KPIs)

  1. First response time (FRT): The average time taken to respond to user inquiries.
  2. Resolution rate: The percentage of tickets resolved within a certain timeframe.
  3. Escalation rate: The number of tickets escalated to senior technicians or management due to complexity or severity.
  4. Customer satisfaction score (CSS): A measure of user satisfaction, calculated by aggregating feedback from surveys and other sources.

Success Metrics for Specific Use Cases

  1. IT service desk: Measure the percentage of issues resolved within a certain timeframe, such as 90% resolution rate within 2 hours.
  2. Customer support: Evaluate the percentage of customer complaints resolved to customers’ satisfaction, such as 85% satisfaction rate with phone support.
  3. Help desk for specific industries (e.g., healthcare): Use industry-specific metrics, such as HIPAA compliance and data security, to assess success.

Metrics for Measuring User Experience

  1. User adoption rate: The percentage of users who adopt the help desk’s services or solutions.
  2. User satisfaction rating: A measure of user satisfaction with the help desk’s performance, calculated by aggregating feedback from surveys and other sources.
  3. Net user experience score (NUES): A measure of overall user experience, calculated by combining multiple metrics, such as response time, resolution rate, and adoption rate.

By tracking these metrics and KPIs, you can evaluate the success of your help desk and identify areas for improvement to enhance user satisfaction and support quality.