Lunch&Learn: Mobile app assessment - how to stand out from the crowd

This week we were delighted to be joined by member Julie Bretland, founder and CEO of Our Mobile Health who kindly hosted a Lunch&Learn on the topic of mobile app assessment. Our Mobile Health (OMH) are the leading experts in digital health app assessment. Their goal is to give healthcare professionals confidence in choosing digital health solutions.

Julie started by painting the landscape for digital health apps which is one of the fastest growing markets in terms of digital.

Screenshot 2018-03-01 at 12.11.36 PM.png

At last count there are over 320,000 health apps on the market so it is a competitive space. Interestingly there are only 85,000 publishers of these apps showing that many publishers have a suite of apps they are trying to find a market for. And it’s a tough business. The top medical app only generated £100,000 from downloads showing that there must be a very long tail of apps making very little revenue. 95% of companies don’t even recover their development costs. In fact Julie explained that many are not getting cash from downloads but from B2B sales and services.

The Challenge

In many aspects of our lives, Julie explained, we seek advice from peers, the press and recommendations. However for medical issues we still defer to our healthcare professionals. When it come to digital health apps, there is a understandable nervousness about which apps to recommend. 

Screenshot 2018-03-01 at 12.15.49 PM.png

A research paper showed that there is an inverse relationship between the popularity of the apps and their effectiveness. There is a huge marketing industry that sits behind getting apps to the top of the app stores. And these apps tend to have great user experience but are maybe not clinically effective.

Julie shared a case study looking at two different heart rate monitors. The new cameras on smartphones can be used to take an elementary heart rate which has spawned a plethora of heart rate monitor apps. However some of these apps require users to give them access to up to 23 data points! 

Contrast this with AliveCor's KardiaMobile smartphone app which uses a medical device bluetoothed to the smartphone to take a medical-grade EKG and in just 30 seconds detect normal heart rhythm or atrial fibrillation (AF).


The app assessment process

Julie then took the audience through the process that they use for assessing mobile apps for the NHS. 

Screenshot 2018-03-01 at 12.38.21 PM.png

OMH look at 10 areas when assessing apps.

  1. An overview of the app and what it does
  2. The effectiveness of the app especially looking at any clinical evidence
  3. If they app has any regulatory approvals and, if not, is it following best practice
  4. The clinical safety of the app
  5. Issues around privacy and confidentiality especially when it comes to patient data
  6. The security of the app, again especially when it comes to patient data
  7. The app’s usability and whether it is accessible for different people
  8. Interoperability     issues
  9. Whether the app is technically stable
  10. How changes and updates to the app and planned and managed.

The review stage is a very important part of the process for OMH. They have a proxy for a peer review by getting three experts look at the app and provide insights into the evidence base.

OMH use the 300 that they developed for the NHS Digital Apps Library to undertake this assessment and it is very much a deep dive into each app rather than a superficial review.

In terms of cost, OMH get paid either by the NHS / charity (see opportunities below) or direct by the app developer. The assessment process costs around £3,200 but can save up to six months development time. 

Julie recommended app developers start to think early about getting ready for assessment long before they plan on submitting for assessment. Start with the 10 areas of assessment on the slide above (all of the questions are on the NHS Digital Apps Library questionnaire.)

Key areas to think about early are:

  • Usability: think about best practice and co-development. If you can show that your have developed your app with users, you are on the right track.
  • Interoperability: structure your data early so it can be interoperable with other systems.
  • Privacy: be very clear on privacy and data security from the start and build in the requirements of the General Data Protection Regulation (GDPR) now.
  • Regulatory: keep up to date with the current regulatory frameworks especially whether your app needs to be classified as a medical device and therefore needs CE marking. Read more about regulation requirements in this blog post from a previous Lunch&Learn.

Current opportunities

To finish the session Julie took us through some current opportunities that makes of digital health apps can engage with now.

NHS Digital Apps Library
OMH are now the official assessors for NHS Digital Apps Library. All apps must pass the Digital Assessment Questionnaire (DAQs). It is a rigorous process with around 300 questions but going through this process will ensure that you are in the OMH family and can be recommended for other opportunities without having to resubmit. If you app is in the public domain, is available in either Apple, Google or Microsoft App Stores and is ready to be assessed, you can find out more here.

EMIS Apps Library
OMH have also they have integrated their app library into EMIS - one of the leading GP systems - and is currently being trialled in North West London. What is exciting about this project is that the apps have been put in the GPs’ workflow and they can recommend apps direct from EMIS. This will be rolled out to 4,000 GPs in April and will form part of the patient record. EMIS will take a revenue share of the app but there may be potential for higher cost apps to be recommended if you can provide it will save GPs time and money down the line. To find out more, contact OMH direct.

NHS Diabetes Prevention Programme – digital stream
The Heathier You: NHS Diabetes Prevention Programme (NHS DPP) was officially launched in 2016 to support people who are at high risk of developing Type 2 diabetes by giving face-to-face tailored support. The digital stream offers similar support but through the use of digital interventions such as apps. 5,000 patients across eight areas in the UK have been recruited and five apps recommended. The programme will give the evidence for prescribing digital apps in 12 months rather than the 4 years it takes to run an RCT.

Good Thinking
Good Thinking is a partnership between the 32 London based NHS Clinical Commissioning Groups, most London Borough Local Authorities, and supported by Public Health England, NHS England and the Mayor of London. The idea is to help Londoners find tools and resources, including mental health apps, to help them and others feel better. To find out more, contact OMH direct.

Parkinson’s UK
Parkinson’s UK, like many health charities, get approached constantly to endorse apps and digital products and services. They are working with OMH to outsource the assessment process and get assessed apps to their community for feedback. As well as apps that can help with Parkinson’s, they are also looking for mental health and different neurological conditions. If you have a relevant app, contact OMH direct.

A big thanks to Julie for sharing her expertise. Do join us for our next Lunch&Learn on 15 March when member Rinesh Amin, co-founder of DrDoctor who is now five years old and is a health tech success story. Rinesh will share the story of how they got through the tough early years without taking outside investment.