Data-Driven EdTech Development

 

Everything is about data these days.

During our EAF 2021 client workshop Riku Alkio from Seppo.io gamification platform, shared their journey as a start-up to break-even situation. When asked, what he wished they would have done differently, his answer revealed lot:

“One thing I would do differently is designing our product for gathering data. Investors are always asking for evidence on impact. This is not always easy to measure and requires deep research, but having data helps. For example, how frequently your users are using your solution, are they using it the way you expect them to, what are the points where they stop using it. " - Riku Alkio, Seppo.io

Seems simple enough: collect data, you might need it later. But how should you plan it?

 

Data for User Experience Optimization

 

The modern way of creating good design and good user experiences relies heavily on measurable testing. This is especially useful, when your design has very clear goals. 

Which registration link placement attracts the best new users? Which wording in your ad gets the most clicks? Base the decision on numbers! Think, if you can compare the change in behavior over time, or can you create A/B test where you compare behavior of two different groups.

There are plenty of analytics tools (such as Amplitude) that can be integrated to give you this data, or you can build them yourself.

Yet, data-based decisions naturally lead to catering for the most common nominator. In the UX community, this has led to some frustration and often legit concern of everything looking the same. And it’s true that leaning on data that reflects the current consensus of users, does not tell what is the norm in the future. Creative design is always a risk, but sometimes it sets new trends.

 

Data for Learning about Your Users' Needs

 

When designing an EdTech product, measuring clicks and use rates of certain functions does not always tell what your users are trying to achieve with your product.  The next step in data utilization is to find trends and patterns with your users. 

For example, the majority of teachers on your EdTech platform could be using it for creating multiple choice quizzes with it, and ignoring your great open-ended exercise creation. This is already valuable information for you, but it doesn’t really tell why this is happening.

Perhaps the teachers find open-ended questions difficult to use. Maybe they use another tool that enables sending video-files or audio comments to students, where your platform only supports text. Or, perhaps they feel your grading system for the questions doesn’t give enough value for the work they need to put in, so they prefer to give assignments in the traditional way.

You could make changes to the product to see if the usage rates improve, but more often you save time by interviewing the users to learn about their real needs and concerns.

 

Data Given to You by Users

 

This is a tricky one.

Are you allowed to scan essays sent by the students for keywords to find out what subjects or topics are important for them? Can you collect geolocation data to see if they are using your service on their way to school or work? Would eye-tracking give you insights of a learner's behavior?  What about data from facial recognition?

Technically this is all possible and data that is produced by the learners, can give great insights on how to personalize and adjust the experience for them. For example: give recommendations of related topics, help by pointing out grammar errors or send push notifications in times when you know they usually engage with your app!  

As with any personal data,  transparency and safe collection and storage of data is crucial. Education sector is a considerable target for cyber attacks, and acts like COPPA or EU GDPR regulate what you can collect; especially if your users are children.  A joint, Erasmus+ project  by EAF, Pålogga and GDRPiS is creating a catalog of GDPR complient Edtech tools.  Companies joining the catalog will be required to fill out a GDPR compliancy assessment questionnaire. This will also help you asses, if your edtech product is controlling or processing information, or for example using profiling, that might compromize your GDPR compliancy. 

Data to Prove Your Impact

 

The main goal of your Edtech solution is not to produce more multiple choice quizzes to the world. It is to help someone learn. Because of this, educational technologies are different from many other digital services. Is the data you get out of your system a proof of learning, or simply proof of usage?

The number of quizzes made on the platform, frequency of login, and even good user ratings on your platforms could simply be a sign that someone finds your solution good enough to be used.

Proof of learning, or learning analytics, are much harder to obtain. Some companies rely on various and complex metrics on user behavior, and claim that the use of AI will eventually solve this puzzle. This approach can be questioned, as Dr Ioanna Noula very clearsightedly does.

Evidence on your impact can be researched differently as your product matures, but even early stage companies should first consider how their product aligns with the learning science principles. All research evidence you gain will grow your impact portfolio.

 

In conclusion

Collect data, you will most likely need it later. But your filing cabinet will be better organized when you define: why the data is collected, and what are you actually measuring with it. Following regulations and keeping your data safe is another major aspect that should be considered right from the start!

Written by Saila Juuti, EAF

This article was written as a part of research conducted for Erasmus+ project on creating a catalog for GDPR compliant teaching tools.