mirror of
https://github.com/MatomoCamp/recording-subtitles.git
synced 2024-09-19 16:03:52 +02:00
2508 lines
67 KiB
Text
2508 lines
67 KiB
Text
1
|
|
00:00:00,000 --> 00:00:07,200
|
|
Hello, everyone.
|
|
|
|
2
|
|
00:00:07,200 --> 00:00:09,280
|
|
Nice to meet you all again.
|
|
|
|
3
|
|
00:00:09,280 --> 00:00:11,480
|
|
It's currently 3.02.
|
|
|
|
4
|
|
00:00:11,480 --> 00:00:15,160
|
|
And we will conduct a conference today.
|
|
|
|
5
|
|
00:00:15,160 --> 00:00:21,920
|
|
So continue by who is here with me and myself.
|
|
|
|
6
|
|
00:00:21,920 --> 00:00:28,320
|
|
So the idea of this conference is to do something which is kind of unusual.
|
|
|
|
7
|
|
00:00:28,320 --> 00:00:30,000
|
|
So we're going to do a use case.
|
|
|
|
8
|
|
00:00:30,000 --> 00:00:35,080
|
|
And most of the time, when a use case is presented by an analyst, it's most of the time, like
|
|
|
|
9
|
|
00:00:35,080 --> 00:00:39,440
|
|
three-quarter of the time, the analyst who is speaking and then a quarter of the time
|
|
|
|
10
|
|
00:00:39,440 --> 00:00:40,440
|
|
with the client.
|
|
|
|
11
|
|
00:00:40,440 --> 00:00:44,520
|
|
Here, this is going to be the reverse mode.
|
|
|
|
12
|
|
00:00:44,520 --> 00:00:51,280
|
|
So I will leave the floor in a couple of minutes to Cathy, who is a client of mine.
|
|
|
|
13
|
|
00:00:51,280 --> 00:00:58,640
|
|
And the idea is that Cathy can introduce you to the project that she's working on.
|
|
|
|
14
|
|
00:00:58,640 --> 00:01:07,640
|
|
And I will explain how I decided to deal with the project management of including Matomo
|
|
|
|
15
|
|
00:01:07,640 --> 00:01:11,800
|
|
within this given project.
|
|
|
|
16
|
|
00:01:11,800 --> 00:01:17,520
|
|
So I leave now the floor to Cathy, and I will interact only dealing with my part of the
|
|
|
|
17
|
|
00:01:17,520 --> 00:01:18,520
|
|
slides.
|
|
|
|
18
|
|
00:01:18,520 --> 00:01:19,520
|
|
OK.
|
|
|
|
19
|
|
00:01:19,520 --> 00:01:20,520
|
|
Thanks, Ronan.
|
|
|
|
20
|
|
00:01:20,520 --> 00:01:21,520
|
|
I'll just share my slides.
|
|
|
|
21
|
|
00:01:21,520 --> 00:01:34,040
|
|
Can I just check that you can see that, Ronan?
|
|
|
|
22
|
|
00:01:34,040 --> 00:01:39,280
|
|
Yes, it looks great.
|
|
|
|
23
|
|
00:01:39,280 --> 00:01:40,280
|
|
OK.
|
|
|
|
24
|
|
00:01:40,280 --> 00:01:41,280
|
|
Lovely.
|
|
|
|
25
|
|
00:01:41,280 --> 00:01:42,280
|
|
Thank you.
|
|
|
|
26
|
|
00:01:42,280 --> 00:01:43,280
|
|
Right.
|
|
|
|
27
|
|
00:01:43,280 --> 00:01:47,920
|
|
So myself and Ronan are going to, as he said, share the presentation today.
|
|
|
|
28
|
|
00:01:47,920 --> 00:01:53,680
|
|
And the presentation is on using Matomo to collect data on intervention engagement within
|
|
|
|
29
|
|
00:01:53,680 --> 00:01:59,880
|
|
a research trial, and the case study is a project called WRAPT.
|
|
|
|
30
|
|
00:01:59,880 --> 00:02:05,880
|
|
So the overview of the presentation is that I'll firstly talk about why I think it's important
|
|
|
|
31
|
|
00:02:05,880 --> 00:02:10,240
|
|
to measure intervention engagement within a research trial.
|
|
|
|
32
|
|
00:02:10,240 --> 00:02:14,040
|
|
I will tell you a bit about the WRAPT research project as well.
|
|
|
|
33
|
|
00:02:14,040 --> 00:02:19,120
|
|
And then Ronan's going to talk about how we use Matomo within the project to collect individual
|
|
|
|
34
|
|
00:02:19,120 --> 00:02:21,800
|
|
level data on engagement.
|
|
|
|
35
|
|
00:02:21,800 --> 00:02:25,320
|
|
Then I'll present some early insights from our analytics data, and then there'll be a
|
|
|
|
36
|
|
00:02:25,320 --> 00:02:30,080
|
|
chance for some questions at the end.
|
|
|
|
37
|
|
00:02:30,080 --> 00:02:35,240
|
|
So first of all, why it's important to measure intervention engagement within a research
|
|
|
|
38
|
|
00:02:35,240 --> 00:02:36,240
|
|
trial.
|
|
|
|
39
|
|
00:02:36,240 --> 00:02:38,000
|
|
Well, there are two reasons.
|
|
|
|
40
|
|
00:02:38,000 --> 00:02:42,040
|
|
One is to minimize something called non-usage attrition.
|
|
|
|
41
|
|
00:02:42,040 --> 00:02:44,760
|
|
And I'll go on to explain a bit about what that is.
|
|
|
|
42
|
|
00:02:44,760 --> 00:02:49,600
|
|
And then also to examine and control for intervention dose.
|
|
|
|
43
|
|
00:02:49,600 --> 00:02:53,080
|
|
So firstly, on to minimizing non-usage attrition.
|
|
|
|
44
|
|
00:02:53,080 --> 00:02:58,920
|
|
So when we are testing whether a new intervention works, then we typically do something called
|
|
|
|
45
|
|
00:02:58,920 --> 00:03:00,640
|
|
a randomized control trial.
|
|
|
|
46
|
|
00:03:00,640 --> 00:03:06,300
|
|
And many of you will be aware of what randomized control trials are already, maybe.
|
|
|
|
47
|
|
00:03:06,300 --> 00:03:08,960
|
|
But they are, I'll just share my next slide.
|
|
|
|
48
|
|
00:03:08,960 --> 00:03:14,680
|
|
So they are used all the time in a clinical context to test the effectiveness of something.
|
|
|
|
49
|
|
00:03:14,680 --> 00:03:20,120
|
|
So that something could be a new drug or a new vaccine, as is the case for the COVID
|
|
|
|
50
|
|
00:03:20,120 --> 00:03:21,120
|
|
vaccine.
|
|
|
|
51
|
|
00:03:21,120 --> 00:03:27,400
|
|
Or in my case, and I'm a health psychologist, I develop interventions that try to encourage
|
|
|
|
52
|
|
00:03:27,400 --> 00:03:29,160
|
|
people to change their health behavior.
|
|
|
|
53
|
|
00:03:29,160 --> 00:03:32,680
|
|
So that could be, for example, to persuade people to use the COVID vaccine.
|
|
|
|
54
|
|
00:03:32,680 --> 00:03:37,000
|
|
So the intervention itself is not a drug or a vaccine or something like that.
|
|
|
|
55
|
|
00:03:37,000 --> 00:03:40,840
|
|
It tends to be a behavior change intervention.
|
|
|
|
56
|
|
00:03:40,840 --> 00:03:43,320
|
|
But the principle of testing it is exactly the same.
|
|
|
|
57
|
|
00:03:43,320 --> 00:03:47,040
|
|
So we would also run a randomized control trial.
|
|
|
|
58
|
|
00:03:47,040 --> 00:03:51,200
|
|
And in randomized control trials, we take a sample population.
|
|
|
|
59
|
|
00:03:51,200 --> 00:03:56,600
|
|
So this is a smaller group of the wider population we're interested in testing the intervention
|
|
|
|
60
|
|
00:03:56,600 --> 00:03:57,600
|
|
on.
|
|
|
|
61
|
|
00:03:57,600 --> 00:04:00,360
|
|
And we'll randomly assign them to two groups.
|
|
|
|
62
|
|
00:04:00,360 --> 00:04:06,040
|
|
So like the flip of a coin or a roll of a dice or more commonly using sort of internet
|
|
|
|
63
|
|
00:04:06,040 --> 00:04:07,560
|
|
based randomization tools.
|
|
|
|
64
|
|
00:04:07,560 --> 00:04:12,120
|
|
But the idea is the same, which is randomly assigning to one group or the other.
|
|
|
|
65
|
|
00:04:12,120 --> 00:04:17,520
|
|
And in the one group, which here we have as group one, all the people in that group that
|
|
|
|
66
|
|
00:04:17,520 --> 00:04:22,640
|
|
have been randomly assigned to it will get the new drug or treatment.
|
|
|
|
67
|
|
00:04:22,640 --> 00:04:27,480
|
|
And then we'll measure afterwards how they fare, what the outcomes were for them.
|
|
|
|
68
|
|
00:04:27,480 --> 00:04:29,640
|
|
And then in the other group, they'll get the control.
|
|
|
|
69
|
|
00:04:29,640 --> 00:04:34,360
|
|
So if it is a drug, it might be something like a sugar pill that has no active ingredients.
|
|
|
|
70
|
|
00:04:34,360 --> 00:04:36,640
|
|
And again, we'll measure the outcome for them.
|
|
|
|
71
|
|
00:04:36,640 --> 00:04:40,520
|
|
And the important point with the randomization is that by the end, we should have balance
|
|
|
|
72
|
|
00:04:40,520 --> 00:04:41,520
|
|
in the two groups.
|
|
|
|
73
|
|
00:04:41,520 --> 00:04:45,960
|
|
So just by chance, we should have approximately even numbers of people of different genders
|
|
|
|
74
|
|
00:04:45,960 --> 00:04:52,440
|
|
and ethnicities or with similar levels of health need or health condition.
|
|
|
|
75
|
|
00:04:52,440 --> 00:04:57,080
|
|
And that's important to make sure that we control for any sort of confounding factors
|
|
|
|
76
|
|
00:04:57,080 --> 00:04:59,320
|
|
that might also have an effect on the outcome.
|
|
|
|
77
|
|
00:04:59,320 --> 00:05:04,880
|
|
And they should then be evenly spread between the two groups.
|
|
|
|
78
|
|
00:05:04,880 --> 00:05:09,080
|
|
So in a clinical context, when a new drug is being tested, adherence to the drug is
|
|
|
|
79
|
|
00:05:09,080 --> 00:05:14,640
|
|
not usually a big problem, although something still to be mindful of.
|
|
|
|
80
|
|
00:05:14,640 --> 00:05:19,840
|
|
And this is because the drug is prescribed by a clinician, and the participants are usually
|
|
|
|
81
|
|
00:05:19,840 --> 00:05:21,400
|
|
very closely supervised.
|
|
|
|
82
|
|
00:05:21,400 --> 00:05:26,760
|
|
So you would be asked to go into a clinical context, for example, and you'd be passed
|
|
|
|
83
|
|
00:05:26,760 --> 00:05:31,880
|
|
the pill on a tray with a cup of water and watched by a research nurse taking it, who
|
|
|
|
84
|
|
00:05:31,880 --> 00:05:36,160
|
|
then recalled that you'd taken it, and that would likely be for every dose.
|
|
|
|
85
|
|
00:05:36,160 --> 00:05:40,600
|
|
And also, as a research participant, you may well, especially if you're in an intervention
|
|
|
|
86
|
|
00:05:40,600 --> 00:05:45,240
|
|
condition, experience observable and immediate health benefits of taking that drug, assuming
|
|
|
|
87
|
|
00:05:45,240 --> 00:05:49,960
|
|
that it's effective, also potentially in the control condition due to placebo effects.
|
|
|
|
88
|
|
00:05:49,960 --> 00:05:56,400
|
|
So there's lots of things that could be meaning that adherence to that drug within an intervention
|
|
|
|
89
|
|
00:05:56,400 --> 00:06:00,440
|
|
and trial context is not too much of a problem.
|
|
|
|
90
|
|
00:06:00,440 --> 00:06:04,280
|
|
It's usually, it's usually reasonably high.
|
|
|
|
91
|
|
00:06:04,280 --> 00:06:11,720
|
|
However, in contract, adherence is a major problem when we're testing e-health interventions.
|
|
|
|
92
|
|
00:06:11,720 --> 00:06:14,360
|
|
So I mentioned briefly about the kind of work that I do.
|
|
|
|
93
|
|
00:06:14,360 --> 00:06:19,920
|
|
So to test health behavior change interventions, and also almost all of the interventions that
|
|
|
|
94
|
|
00:06:19,920 --> 00:06:22,460
|
|
I develop and test are of the e-health type.
|
|
|
|
95
|
|
00:06:22,460 --> 00:06:25,400
|
|
So they're interventions delivered via the internet.
|
|
|
|
96
|
|
00:06:25,400 --> 00:06:32,360
|
|
So here, the intervention is neither prescribed nor critical to well-being.
|
|
|
|
97
|
|
00:06:32,360 --> 00:06:41,000
|
|
And this graph, I think, demonstrates quite nicely typical attrition from e-health interventions.
|
|
|
|
98
|
|
00:06:41,000 --> 00:06:46,320
|
|
So on the y-axis going up the left, you can see at the very top, there's a 1, and that
|
|
|
|
99
|
|
00:06:46,320 --> 00:06:53,080
|
|
reflects 100% of users of the intervention going down to 90, 80, 70, et cetera, right
|
|
|
|
100
|
|
00:06:53,080 --> 00:06:54,960
|
|
down to 0 at the bottom.
|
|
|
|
101
|
|
00:06:54,960 --> 00:06:59,800
|
|
And then across the bottom on the x-axis, we've got time and modules.
|
|
|
|
102
|
|
00:06:59,800 --> 00:07:08,200
|
|
So if we take the blue line to start with, the study by Farrell Dunn, this was a 12-week
|
|
|
|
103
|
|
00:07:08,200 --> 00:07:12,920
|
|
program of internet delivery of an intervention.
|
|
|
|
104
|
|
00:07:12,920 --> 00:07:17,920
|
|
So it's supposed to engage for 12 weeks in total to have had the full dose of the intervention.
|
|
|
|
105
|
|
00:07:17,920 --> 00:07:22,740
|
|
But you can see after a couple of weeks, we've already got less than 50% of people still
|
|
|
|
106
|
|
00:07:22,740 --> 00:07:29,080
|
|
using that website, going down to less than 20% at week 3.
|
|
|
|
107
|
|
00:07:29,080 --> 00:07:33,200
|
|
And you can see by the end, in fact, it was less than 1% of people still using or still
|
|
|
|
108
|
|
00:07:33,200 --> 00:07:37,480
|
|
completing the intervention right at the very end.
|
|
|
|
109
|
|
00:07:37,480 --> 00:07:43,240
|
|
And then for the dashed black line, this was an intervention called Mood Gym.
|
|
|
|
110
|
|
00:07:43,240 --> 00:07:44,880
|
|
And it's in a trial context again.
|
|
|
|
111
|
|
00:07:44,880 --> 00:07:47,920
|
|
And there were five modules here for this intervention.
|
|
|
|
112
|
|
00:07:47,920 --> 00:07:54,440
|
|
You can see that less than 70% did module 2, and then module 3, less than 60%, and so
|
|
|
|
113
|
|
00:07:54,440 --> 00:07:58,520
|
|
on, down to just over 20% doing all five modules.
|
|
|
|
114
|
|
00:07:58,520 --> 00:08:01,680
|
|
And then the solid black line is also Mood Gym.
|
|
|
|
115
|
|
00:08:01,680 --> 00:08:04,920
|
|
But in this context, it wasn't in a trial.
|
|
|
|
116
|
|
00:08:04,920 --> 00:08:09,100
|
|
It was just people allowed to have access to it, then see who engaged with it and carried
|
|
|
|
117
|
|
00:08:09,100 --> 00:08:11,640
|
|
on engaging with it.
|
|
|
|
118
|
|
00:08:11,640 --> 00:08:16,680
|
|
So outside the context of a trial in which people might be incentivized to use the intervention
|
|
|
|
119
|
|
00:08:16,680 --> 00:08:22,720
|
|
and also encouraged and reminded by researchers to do so, you can see that engagement is even
|
|
|
|
120
|
|
00:08:22,720 --> 00:08:26,200
|
|
less.
|
|
|
|
121
|
|
00:08:26,200 --> 00:08:33,320
|
|
So non-use of nutrition, so participants ceasing to use the intervention in a trial context,
|
|
|
|
122
|
|
00:08:33,320 --> 00:08:37,200
|
|
is obviously a big problem for us.
|
|
|
|
123
|
|
00:08:37,200 --> 00:08:41,880
|
|
And if participants are not using intervention in a trial context, it makes it really difficult
|
|
|
|
124
|
|
00:08:41,880 --> 00:08:46,080
|
|
to test whether the intervention actually works or not.
|
|
|
|
125
|
|
00:08:46,080 --> 00:08:55,200
|
|
So in trials, regardless of, sorry, in trials, the outcome data for every participant is
|
|
|
|
126
|
|
00:08:55,200 --> 00:08:57,760
|
|
randomized and included in the analysis.
|
|
|
|
127
|
|
00:08:57,760 --> 00:09:04,200
|
|
So if we have people in that, if you remember that first diagram I showed you of a randomized
|
|
|
|
128
|
|
00:09:04,200 --> 00:09:05,200
|
|
controls trial.
|
|
|
|
129
|
|
00:09:05,200 --> 00:09:09,160
|
|
So we've got people randomized to the two groups, and in that top arm, we've got people
|
|
|
|
130
|
|
00:09:09,160 --> 00:09:11,480
|
|
who received the intervention itself.
|
|
|
|
131
|
|
00:09:11,480 --> 00:09:15,100
|
|
If you've only got a small proportion of those who are actually using intervention of four,
|
|
|
|
132
|
|
00:09:15,100 --> 00:09:19,760
|
|
you might be thinking to yourself, well, just analyze data from those people who use the
|
|
|
|
133
|
|
00:09:19,760 --> 00:09:23,480
|
|
intervention itself and ignore everyone else who didn't.
|
|
|
|
134
|
|
00:09:23,480 --> 00:09:27,720
|
|
But that threatens the whole integrity of the trial design, which is the randomization
|
|
|
|
135
|
|
00:09:27,720 --> 00:09:29,760
|
|
of people to one group or the other.
|
|
|
|
136
|
|
00:09:29,760 --> 00:09:34,240
|
|
I explained earlier about how we randomize so we have balance between the two arms.
|
|
|
|
137
|
|
00:09:34,240 --> 00:09:39,800
|
|
But if you're only including a subgroup of those people who have used the intervention
|
|
|
|
138
|
|
00:09:39,800 --> 00:09:44,480
|
|
to a minimum extent and ignoring everybody else, then you remove that balance, which
|
|
|
|
139
|
|
00:09:44,480 --> 00:09:51,800
|
|
is so important to that trial validity.
|
|
|
|
140
|
|
00:09:51,800 --> 00:09:59,680
|
|
So the point of ensuring that everybody, regardless of whether they use the intervention or not
|
|
|
|
141
|
|
00:09:59,680 --> 00:10:02,440
|
|
in a trial, it's called intention to treat analysis.
|
|
|
|
142
|
|
00:10:02,440 --> 00:10:07,920
|
|
So if you randomize someone, you always analyze their data regardless to ensure you've got
|
|
|
|
143
|
|
00:10:07,920 --> 00:10:09,880
|
|
this balance in design.
|
|
|
|
144
|
|
00:10:09,880 --> 00:10:13,760
|
|
But the problem, of course, is if you've got a group of people in your intervention condition
|
|
|
|
145
|
|
00:10:13,760 --> 00:10:17,520
|
|
that haven't used the intervention at all or not very much, then if your intervention
|
|
|
|
146
|
|
00:10:17,520 --> 00:10:22,240
|
|
is indeed effective, then when you do the analysis, it has the effect of underestimating
|
|
|
|
147
|
|
00:10:22,240 --> 00:10:25,960
|
|
the effect of the intervention because you're including in the analysis in that intervention
|
|
|
|
148
|
|
00:10:25,960 --> 00:10:30,600
|
|
condition people who didn't even use it at all or didn't use it very much at all.
|
|
|
|
149
|
|
00:10:30,600 --> 00:10:36,120
|
|
And remember that diagram of it dropping off steeply, that is often the case for us.
|
|
|
|
150
|
|
00:10:36,120 --> 00:10:40,960
|
|
So it's really, really important that in a trial context that we do absolutely everything
|
|
|
|
151
|
|
00:10:40,960 --> 00:10:49,120
|
|
that we can to minimize people not using that intervention or dropping off.
|
|
|
|
152
|
|
00:10:49,120 --> 00:10:54,760
|
|
Now ahead of doing a main trial in which we test the effectiveness of the kind of intervention
|
|
|
|
153
|
|
00:10:54,760 --> 00:10:59,000
|
|
that I've been talking about, we tend to do something called a feasibility randomized
|
|
|
|
154
|
|
00:10:59,000 --> 00:11:03,760
|
|
control trial, which is exactly the same as a randomized control trial with this randomization
|
|
|
|
155
|
|
00:11:03,760 --> 00:11:05,160
|
|
to the two groups.
|
|
|
|
156
|
|
00:11:05,160 --> 00:11:08,680
|
|
The only thing that's different is it's with much fewer participants.
|
|
|
|
157
|
|
00:11:08,680 --> 00:11:12,760
|
|
And the purpose of that kind of study is to help us to prepare for a full randomized control
|
|
|
|
158
|
|
00:11:12,760 --> 00:11:13,760
|
|
trial.
|
|
|
|
159
|
|
00:11:13,760 --> 00:11:18,680
|
|
And what we're trying to do is resolve or remove all the unknowns.
|
|
|
|
160
|
|
00:11:18,680 --> 00:11:24,200
|
|
So for example, to help us to learn what the best methods for recruiting participants is,
|
|
|
|
161
|
|
00:11:24,200 --> 00:11:27,920
|
|
what level of incentive we need to recruit people in the first place, but also to keep
|
|
|
|
162
|
|
00:11:27,920 --> 00:11:29,240
|
|
them in the trial until the end.
|
|
|
|
163
|
|
00:11:29,240 --> 00:11:33,320
|
|
So all those kinds of things are important for us to establish in this early phase before
|
|
|
|
164
|
|
00:11:33,320 --> 00:11:36,080
|
|
we do the main thing.
|
|
|
|
165
|
|
00:11:36,080 --> 00:11:44,160
|
|
But we can also use these feasibility randomized control trials, these preparation type trials,
|
|
|
|
166
|
|
00:11:44,160 --> 00:11:48,320
|
|
is to examine non-nucid attrition, which we've been talking about.
|
|
|
|
167
|
|
00:11:48,320 --> 00:11:53,600
|
|
So if we look at attrition in this early stage trial, it can tell us some important things
|
|
|
|
168
|
|
00:11:53,600 --> 00:11:58,040
|
|
such as whether attrition is associated with particular demographic factors.
|
|
|
|
169
|
|
00:11:58,040 --> 00:12:01,560
|
|
And of course, we're measuring those in the surveys that we give to people before they
|
|
|
|
170
|
|
00:12:01,560 --> 00:12:03,840
|
|
have the intervention and then also afterwards.
|
|
|
|
171
|
|
00:12:03,840 --> 00:12:09,400
|
|
And so if we can link that demographic data with the data about attrition, we can learn
|
|
|
|
172
|
|
00:12:09,400 --> 00:12:10,520
|
|
some important things.
|
|
|
|
173
|
|
00:12:10,520 --> 00:12:14,720
|
|
So if we found out that, for example, females in particular weren't using our intervention
|
|
|
|
174
|
|
00:12:14,720 --> 00:12:19,160
|
|
or dropping off, then it would be important to maybe interview them as part of some later
|
|
|
|
175
|
|
00:12:19,160 --> 00:12:24,800
|
|
studies to find out why that might be the case and to try and rectify that.
|
|
|
|
176
|
|
00:12:24,800 --> 00:12:28,600
|
|
And when we look at the shape of that attrition curve, which we were just looking at before,
|
|
|
|
177
|
|
00:12:28,600 --> 00:12:32,600
|
|
that can tell us something also important.
|
|
|
|
178
|
|
00:12:32,600 --> 00:12:38,320
|
|
So if we had, should I get my mouse to do that?
|
|
|
|
179
|
|
00:12:38,320 --> 00:12:39,320
|
|
Yeah.
|
|
|
|
180
|
|
00:12:39,320 --> 00:12:44,400
|
|
So if we had a shape a bit sort of like really steep down like that, that would be a really
|
|
|
|
181
|
|
00:12:44,400 --> 00:12:49,840
|
|
bad sign for our intervention, tell us that people were disengaging very quickly because
|
|
|
|
182
|
|
00:12:49,840 --> 00:12:52,080
|
|
they just didn't like it.
|
|
|
|
183
|
|
00:12:52,080 --> 00:12:59,200
|
|
And then this more kind of sigmoid style curve would tell us that we had some sort of initial
|
|
|
|
184
|
|
00:12:59,200 --> 00:13:03,560
|
|
interest in it, which then dropped off, as you might imagine, then a bit of a steep drop
|
|
|
|
185
|
|
00:13:03,560 --> 00:13:08,120
|
|
off in the middle, perhaps when there's less motivated people were leaving or possibly
|
|
|
|
186
|
|
00:13:08,120 --> 00:13:10,600
|
|
representing some sort of usability style problems.
|
|
|
|
187
|
|
00:13:10,600 --> 00:13:14,520
|
|
And then we've got a hardcore sort of set of users at the bottom.
|
|
|
|
188
|
|
00:13:14,520 --> 00:13:19,360
|
|
So the style and shape, I suppose, of this curve can tell us some important things as
|
|
|
|
189
|
|
00:13:19,360 --> 00:13:24,680
|
|
well if we're measuring attrition from our intervention during that feasibility randomized
|
|
|
|
190
|
|
00:13:24,680 --> 00:13:28,840
|
|
control trial.
|
|
|
|
191
|
|
00:13:28,840 --> 00:13:35,480
|
|
So it's not uncommon for feasibility randomized control trials of e-health interventions to
|
|
|
|
192
|
|
00:13:35,480 --> 00:13:42,000
|
|
measure non-nucid attrition, but measures of engagement are usually quite limited, they're
|
|
|
|
193
|
|
00:13:42,000 --> 00:13:43,860
|
|
not so fine grained.
|
|
|
|
194
|
|
00:13:43,860 --> 00:13:50,960
|
|
So often it's about when people registering within the website itself, so we know who's
|
|
|
|
195
|
|
00:13:50,960 --> 00:13:54,720
|
|
visiting and then we know when they stopped visiting and we might also be able to collect
|
|
|
|
196
|
|
00:13:54,720 --> 00:13:58,520
|
|
data on whether they've completed modules within that website itself, but that tends
|
|
|
|
197
|
|
00:13:58,520 --> 00:14:03,320
|
|
to be the limitation or the limits of it.
|
|
|
|
198
|
|
00:14:03,320 --> 00:14:07,960
|
|
And the use of analytics data rarely happens at all, but I've sort of become increasingly
|
|
|
|
199
|
|
00:14:07,960 --> 00:14:13,320
|
|
aware recently that analytics data can offer much greater insight into usability issues.
|
|
|
|
200
|
|
00:14:13,320 --> 00:14:19,960
|
|
For example, are the particular points during the different stages in which people are going
|
|
|
|
201
|
|
00:14:19,960 --> 00:14:25,400
|
|
through the intervention where they're experiencing some technical issues, it's not very nice,
|
|
|
|
202
|
|
00:14:25,400 --> 00:14:28,600
|
|
the look and feel of people are dropping off because they're having problems with it.
|
|
|
|
203
|
|
00:14:28,600 --> 00:14:31,320
|
|
And these are the kind of things that can be rectified, which if we don't collect this
|
|
|
|
204
|
|
00:14:31,320 --> 00:14:35,480
|
|
kind of data, we won't know anything about.
|
|
|
|
205
|
|
00:14:35,480 --> 00:14:39,560
|
|
And typically within feasibility randomized control trials, we tend to have a stage of
|
|
|
|
206
|
|
00:14:39,560 --> 00:14:43,360
|
|
qualitative research, so we do interviews with our participants to find out more about
|
|
|
|
207
|
|
00:14:43,360 --> 00:14:47,840
|
|
their experience of being a research participant and what we can learn to improve the trial.
|
|
|
|
208
|
|
00:14:47,840 --> 00:14:52,560
|
|
And this would be a great stage at which to ask people to, so we can draw hypotheses from
|
|
|
|
209
|
|
00:14:52,560 --> 00:14:56,760
|
|
these usability issues and then develop them further through those interviews to find out
|
|
|
|
210
|
|
00:14:56,760 --> 00:14:58,160
|
|
more about people's experiences.
|
|
|
|
211
|
|
00:14:58,160 --> 00:15:03,960
|
|
So those two things would dovetail really nicely together.
|
|
|
|
212
|
|
00:15:03,960 --> 00:15:10,920
|
|
So yeah, we can do these feasibility randomized control trials and examine usability issues
|
|
|
|
213
|
|
00:15:10,920 --> 00:15:16,960
|
|
with a view to properly preparing a much better and resolving any of these issues in preparation
|
|
|
|
214
|
|
00:15:16,960 --> 00:15:18,440
|
|
for the main trial.
|
|
|
|
215
|
|
00:15:18,440 --> 00:15:24,640
|
|
And I just want to make the point that I'm very aware that within the tech sort of web
|
|
|
|
216
|
|
00:15:24,640 --> 00:15:32,680
|
|
industry sector, that usability and examining the sort of client or customer's use of websites
|
|
|
|
217
|
|
00:15:32,680 --> 00:15:36,720
|
|
and all these friction points and things that are going wrong in order to maximize engagement
|
|
|
|
218
|
|
00:15:36,720 --> 00:15:41,640
|
|
and getting people through to the end point, whatever that might be, like purchasing something,
|
|
|
|
219
|
|
00:15:41,640 --> 00:15:44,480
|
|
is done really commonly and very well.
|
|
|
|
220
|
|
00:15:44,480 --> 00:15:49,160
|
|
But really, in my field, it's rarely done at all, not very much at all.
|
|
|
|
221
|
|
00:15:49,160 --> 00:15:53,400
|
|
And I just know there's a huge amount to be learned and gained from collaborating with
|
|
|
|
222
|
|
00:15:53,400 --> 00:15:56,920
|
|
people who know much about this, and I'd be really interested in hearing from somebody
|
|
|
|
223
|
|
00:15:56,920 --> 00:16:03,720
|
|
who would be interested in working with me in that kind of way.
|
|
|
|
224
|
|
00:16:03,720 --> 00:16:11,000
|
|
So yeah, I've hopefully sort of made the case for why it's important to look at non user
|
|
|
|
225
|
|
00:16:11,000 --> 00:16:16,080
|
|
attrition and for minimizing that in the trial, we know it's important to do that.
|
|
|
|
226
|
|
00:16:16,080 --> 00:16:19,720
|
|
And running these types of feasibility randomized control trials, which are really common, is
|
|
|
|
227
|
|
00:16:19,720 --> 00:16:25,320
|
|
a great opportunity to look at attrition and try and remedy it and do something about it.
|
|
|
|
228
|
|
00:16:25,320 --> 00:16:28,960
|
|
And I'm really, and everyone else in my field, just at the beginning of trying to understand
|
|
|
|
229
|
|
00:16:28,960 --> 00:16:34,240
|
|
how best to measure this type of attrition, to understand what all the measures and things
|
|
|
|
230
|
|
00:16:34,240 --> 00:16:38,840
|
|
that we can collect are telling us, and also to make the right sort of changes so that
|
|
|
|
231
|
|
00:16:38,840 --> 00:16:42,480
|
|
in the main trial, we can actually have as high an attrition as possible.
|
|
|
|
232
|
|
00:16:42,480 --> 00:16:46,480
|
|
I mean, if people don't like the intervention, that's another thing we can't, that's a different
|
|
|
|
233
|
|
00:16:46,480 --> 00:16:47,480
|
|
question to be answered.
|
|
|
|
234
|
|
00:16:47,480 --> 00:16:51,200
|
|
But if it's to do with how the website itself is operating, I think there's lots we can
|
|
|
|
235
|
|
00:16:51,200 --> 00:16:53,680
|
|
do about that.
|
|
|
|
236
|
|
00:16:53,680 --> 00:16:57,320
|
|
And then the other issue, and I've only got two slides on this one, this is a much smaller
|
|
|
|
237
|
|
00:16:57,320 --> 00:17:05,100
|
|
point really, but equally important is around examining and controlling for dose.
|
|
|
|
238
|
|
00:17:05,100 --> 00:17:11,280
|
|
So the amount or dose of an intervention that a participant receives in a trial can affect
|
|
|
|
239
|
|
00:17:11,280 --> 00:17:13,120
|
|
the strength of the outcome.
|
|
|
|
240
|
|
00:17:13,120 --> 00:17:18,280
|
|
So again, back to that randomization that we talked about, so those in the top arm who
|
|
|
|
241
|
|
00:17:18,280 --> 00:17:21,900
|
|
received the intervention, so we look to see what the outcomes are.
|
|
|
|
242
|
|
00:17:21,900 --> 00:17:26,880
|
|
So if it's a stop smoking intervention, for example, we'll measure number of cigarettes
|
|
|
|
243
|
|
00:17:26,880 --> 00:17:31,080
|
|
per day that I had before the intervention and afterwards, and then we'll compare the
|
|
|
|
244
|
|
00:17:31,080 --> 00:17:34,800
|
|
two arms to look at the number of cigarettes smoked.
|
|
|
|
245
|
|
00:17:34,800 --> 00:17:39,440
|
|
So we need to know the dose of that intervention.
|
|
|
|
246
|
|
00:17:39,440 --> 00:17:43,440
|
|
So if it's a web intervention, we stop smoking, we need to know the amount of consumption
|
|
|
|
247
|
|
00:17:43,440 --> 00:17:44,440
|
|
of that website.
|
|
|
|
248
|
|
00:17:44,440 --> 00:17:45,680
|
|
Have they gone back a lot every day?
|
|
|
|
249
|
|
00:17:45,680 --> 00:17:49,240
|
|
Have they reviewed everything, ordered everything they can, that sort of thing, to know the
|
|
|
|
250
|
|
00:17:49,240 --> 00:17:50,240
|
|
dose.
|
|
|
|
251
|
|
00:17:50,240 --> 00:17:52,720
|
|
And that way we can control for the analysis.
|
|
|
|
252
|
|
00:17:52,720 --> 00:17:58,000
|
|
So we can just isolate the effect of the intervention and not the amount they're having, but then
|
|
|
|
253
|
|
00:17:58,000 --> 00:18:02,680
|
|
also we can measure the correlation or the strength of relationship between dose and
|
|
|
|
254
|
|
00:18:02,680 --> 00:18:03,680
|
|
outcome as well.
|
|
|
|
255
|
|
00:18:03,680 --> 00:18:09,060
|
|
These are really important for us to understand, but you can't examine what you don't measure.
|
|
|
|
256
|
|
00:18:09,060 --> 00:18:12,560
|
|
So we really need to understand what is the dose, the amount that people are having of
|
|
|
|
257
|
|
00:18:12,560 --> 00:18:19,000
|
|
each of our interventions, and this is to do with the main trial itself.
|
|
|
|
258
|
|
00:18:19,000 --> 00:18:26,320
|
|
So if we can measure dose and for this, it's really important that the outcomes are linked
|
|
|
|
259
|
|
00:18:26,320 --> 00:18:29,700
|
|
to each individual participant in the study.
|
|
|
|
260
|
|
00:18:29,700 --> 00:18:34,480
|
|
So if we can measure dose, then we can understand things such as, is there a linear relationship
|
|
|
|
261
|
|
00:18:34,480 --> 00:18:36,200
|
|
between dose and outcome?
|
|
|
|
262
|
|
00:18:36,200 --> 00:18:41,120
|
|
So in other words, the more of the intervention that somebody has, the greater the benefit.
|
|
|
|
263
|
|
00:18:41,120 --> 00:18:44,840
|
|
Or is it there are non-linear relationships at the point of saturation, which doesn't
|
|
|
|
264
|
|
00:18:44,840 --> 00:18:47,420
|
|
really make any difference whether you have any more or not?
|
|
|
|
265
|
|
00:18:47,420 --> 00:18:52,840
|
|
Or is there minimum dose that everyone needs to have in order for an effect to be achieved?
|
|
|
|
266
|
|
00:18:52,840 --> 00:18:58,560
|
|
And analytics data can also be used for this and can provide us with really precise and
|
|
|
|
267
|
|
00:18:58,560 --> 00:19:04,240
|
|
detailed measures of the dose for each individual, such as which pages are accessed, how long
|
|
|
|
268
|
|
00:19:04,240 --> 00:19:10,720
|
|
people spend on each page, which videos have been watched, and for how long.
|
|
|
|
269
|
|
00:19:10,720 --> 00:19:18,240
|
|
So I'd now like to go on telling a little bit about an intervention that I developed
|
|
|
|
270
|
|
00:19:18,240 --> 00:19:19,240
|
|
called RAPT.
|
|
|
|
271
|
|
00:19:19,240 --> 00:19:25,440
|
|
And I say I, not me alone, me with a fantastic team of other researchers.
|
|
|
|
272
|
|
00:19:25,440 --> 00:19:29,320
|
|
So first, just say a little bit about the study we're doing.
|
|
|
|
273
|
|
00:19:29,320 --> 00:19:33,680
|
|
So we're at the stage of doing the feasibility randomized control trial, so the preparatory
|
|
|
|
274
|
|
00:19:33,680 --> 00:19:36,360
|
|
study before the main trial.
|
|
|
|
275
|
|
00:19:36,360 --> 00:19:42,560
|
|
And this aims to establish if the main randomized control trial is feasible and to inform our
|
|
|
|
276
|
|
00:19:42,560 --> 00:19:43,560
|
|
preparations.
|
|
|
|
277
|
|
00:19:43,560 --> 00:19:49,440
|
|
And we are using analytics data to better understand any possible usability issues so
|
|
|
|
278
|
|
00:19:49,440 --> 00:19:53,840
|
|
that they can be addressed ahead of the trial, so to minimize the non-usage attrition that
|
|
|
|
279
|
|
00:19:53,840 --> 00:19:54,840
|
|
I talked about.
|
|
|
|
280
|
|
00:19:54,840 --> 00:19:59,440
|
|
But also to learn how best to measure dose in preparation for our main randomized control
|
|
|
|
281
|
|
00:19:59,440 --> 00:20:03,400
|
|
trial.
|
|
|
|
282
|
|
00:20:03,400 --> 00:20:08,040
|
|
So the RAPT intervention itself, I'm going to show you in a minute what the website looks
|
|
|
|
283
|
|
00:20:08,040 --> 00:20:11,040
|
|
like, but just a little bit of information about it.
|
|
|
|
284
|
|
00:20:11,040 --> 00:20:16,900
|
|
So it aims to increase condom use by addressing factors such as people's attitudes towards
|
|
|
|
285
|
|
00:20:16,900 --> 00:20:20,560
|
|
condoms or whether they feel positively or negatively about them and the beliefs that
|
|
|
|
286
|
|
00:20:20,560 --> 00:20:26,440
|
|
underlie that, people's self-efficacy for communicating about condoms and their use.
|
|
|
|
287
|
|
00:20:26,440 --> 00:20:31,600
|
|
So this is about, broadly speaking, how confident you feel to raise condom use with a partner
|
|
|
|
288
|
|
00:20:31,600 --> 00:20:37,800
|
|
for the first time and also how confident you feel in using them correctly.
|
|
|
|
289
|
|
00:20:37,800 --> 00:20:43,640
|
|
And it's also about increasing access to condoms, so about making them more available to people
|
|
|
|
290
|
|
00:20:43,640 --> 00:20:47,680
|
|
through a condom distribution scheme.
|
|
|
|
291
|
|
00:20:47,680 --> 00:20:53,000
|
|
And there are six components in total which are tailored to individual need.
|
|
|
|
292
|
|
00:20:53,000 --> 00:20:58,480
|
|
So at the very start of access to the website, we ask our users to answer some very quick
|
|
|
|
293
|
|
00:20:58,480 --> 00:21:02,640
|
|
yes-no questions about their main barriers to condom use, and then they get allocated
|
|
|
|
294
|
|
00:21:02,640 --> 00:21:05,500
|
|
between one and six of the components.
|
|
|
|
295
|
|
00:21:05,500 --> 00:21:11,800
|
|
So the intervention itself, hopefully, is quite closely aligned to their needs and interests.
|
|
|
|
296
|
|
00:21:11,800 --> 00:21:16,600
|
|
And there are three products which can be ordered, a trial pack of condoms, a condom
|
|
|
|
297
|
|
00:21:16,600 --> 00:21:22,400
|
|
carrier, and access to a service which supplies condoms on a monthly basis.
|
|
|
|
298
|
|
00:21:22,400 --> 00:21:25,880
|
|
And there are also three lots of videos.
|
|
|
|
299
|
|
00:21:25,880 --> 00:21:40,320
|
|
So I'm just going to change my screen to show you the website itself.
|
|
|
|
300
|
|
00:21:40,320 --> 00:21:43,280
|
|
So I'm going to just shout if you can't see this, otherwise I'll just show you that everybody
|
|
|
|
301
|
|
00:21:43,280 --> 00:21:44,280
|
|
can.
|
|
|
|
302
|
|
00:21:44,280 --> 00:21:47,240
|
|
So here is the wrapped website.
|
|
|
|
303
|
|
00:21:47,240 --> 00:21:50,400
|
|
And after you've answered those few quick questions at the start, this is what you will
|
|
|
|
304
|
|
00:21:50,400 --> 00:21:51,800
|
|
see.
|
|
|
|
305
|
|
00:21:51,800 --> 00:21:56,000
|
|
And this person here has been assigned all six intervention components.
|
|
|
|
306
|
|
00:21:56,000 --> 00:21:59,920
|
|
And these are represented by these blocks on the screen here.
|
|
|
|
307
|
|
00:21:59,920 --> 00:22:04,680
|
|
So I won't show you all of it, but I'll just give you a bit of a flavor for it.
|
|
|
|
308
|
|
00:22:04,680 --> 00:22:09,120
|
|
So with the sample pack here, if we click on that.
|
|
|
|
309
|
|
00:22:09,120 --> 00:22:12,360
|
|
So here, people get the chance to customize the pack.
|
|
|
|
310
|
|
00:22:12,360 --> 00:22:14,600
|
|
It's a box with a tray.
|
|
|
|
311
|
|
00:22:14,600 --> 00:22:19,120
|
|
And then they can choose the color and an insert that goes inside it.
|
|
|
|
312
|
|
00:22:19,120 --> 00:22:27,120
|
|
And then they get to, I just, yeah, choose that.
|
|
|
|
313
|
|
00:22:27,120 --> 00:22:28,640
|
|
And then they get to order it.
|
|
|
|
314
|
|
00:22:28,640 --> 00:22:36,280
|
|
And it has 12 different types of condoms and three sachets of flub inside there.
|
|
|
|
315
|
|
00:22:36,280 --> 00:22:39,240
|
|
And then we've got videos as well.
|
|
|
|
316
|
|
00:22:39,240 --> 00:22:47,920
|
|
So this one is about a video, and this is of young people demoing how to put on a condom
|
|
|
|
317
|
|
00:22:47,920 --> 00:22:49,880
|
|
correctly without any errors.
|
|
|
|
318
|
|
00:22:49,880 --> 00:22:53,960
|
|
And there are various other videos as one of the things to order.
|
|
|
|
319
|
|
00:22:53,960 --> 00:22:59,000
|
|
That just gives you a bit of a flavor of the website and what it looks like.
|
|
|
|
320
|
|
00:22:59,000 --> 00:23:03,920
|
|
So I'll just go back to the presentation again.
|
|
|
|
321
|
|
00:23:03,920 --> 00:23:13,800
|
|
So in terms of data collection, the study requires participants in our feasibility randomized
|
|
|
|
322
|
|
00:23:13,800 --> 00:23:17,520
|
|
control trial to complete activities over a 12-month period.
|
|
|
|
323
|
|
00:23:17,520 --> 00:23:21,240
|
|
So they consent to the study and complete a survey.
|
|
|
|
324
|
|
00:23:21,240 --> 00:23:25,160
|
|
And then they get directed to either, well, they're randomized.
|
|
|
|
325
|
|
00:23:25,160 --> 00:23:29,240
|
|
They go into one arm, which is to receive the wrapped intervention website I've just
|
|
|
|
326
|
|
00:23:29,240 --> 00:23:33,560
|
|
shown you, or they get randomized to a control website, which has the same branding and sort
|
|
|
|
327
|
|
00:23:33,560 --> 00:23:39,400
|
|
of color and logos, but has very basic static information on condom use and sexually transmitted
|
|
|
|
328
|
|
00:23:39,400 --> 00:23:41,200
|
|
infections.
|
|
|
|
329
|
|
00:23:41,200 --> 00:23:49,040
|
|
And then they get sent another survey three months later, and also a test for an STI called
|
|
|
|
330
|
|
00:23:49,040 --> 00:23:55,320
|
|
chlamydia, which gets sent, like a test gets sent in the post to them to complete.
|
|
|
|
331
|
|
00:23:55,320 --> 00:23:59,480
|
|
And then at six months, another survey, and then at 12 months, another survey, and also
|
|
|
|
332
|
|
00:23:59,480 --> 00:24:03,680
|
|
the same test for an STI.
|
|
|
|
333
|
|
00:24:03,680 --> 00:24:07,920
|
|
So they've been asked to do quite a lot from us, and they're incentivized to complete those
|
|
|
|
334
|
|
00:24:07,920 --> 00:24:09,720
|
|
different activities.
|
|
|
|
335
|
|
00:24:09,720 --> 00:24:15,720
|
|
And we're using some database management software called RedCap to consent participants to prompt
|
|
|
|
336
|
|
00:24:15,720 --> 00:24:20,280
|
|
them to complete all those different activities that I've just told you about and to record
|
|
|
|
337
|
|
00:24:20,280 --> 00:24:24,720
|
|
their data so it sends out the surveys to them and we record their test results for
|
|
|
|
338
|
|
00:24:24,720 --> 00:24:26,720
|
|
the STIs in there as well.
|
|
|
|
339
|
|
00:24:26,720 --> 00:24:33,440
|
|
Now, RedCap used to direct participants to the two different websites, so there's an
|
|
|
|
340
|
|
00:24:33,440 --> 00:24:37,840
|
|
automated email that goes out to people after they've been randomized, which has a link
|
|
|
|
341
|
|
00:24:37,840 --> 00:24:41,160
|
|
into one of the two different websites, and they have to click on that, and then they'll
|
|
|
|
342
|
|
00:24:41,160 --> 00:24:44,720
|
|
go to each of the two different websites.
|
|
|
|
343
|
|
00:24:44,720 --> 00:24:51,680
|
|
And every activity that any participant in our study does is linked to a unique ID, so
|
|
|
|
344
|
|
00:24:51,680 --> 00:24:55,440
|
|
everything they do is logged against that unique ID.
|
|
|
|
345
|
|
00:24:55,440 --> 00:25:02,760
|
|
So we were going to be running this feasibility randomized control trial, and we knew that
|
|
|
|
346
|
|
00:25:02,760 --> 00:25:09,000
|
|
we wanted to be able to measure attrition both broadly on an aggregate level, but also
|
|
|
|
347
|
|
00:25:09,000 --> 00:25:12,920
|
|
individually, so we could look at things like demographic data and how that links to attrition,
|
|
|
|
348
|
|
00:25:12,920 --> 00:25:18,160
|
|
but then also to work out how best to measure dose for our main trial so that we can understand
|
|
|
|
349
|
|
00:25:18,160 --> 00:25:21,960
|
|
those important things about how dose relates to outcomes.
|
|
|
|
350
|
|
00:25:21,960 --> 00:25:24,160
|
|
But we really didn't know how to go about doing this.
|
|
|
|
351
|
|
00:25:24,160 --> 00:25:29,880
|
|
We wanted something more fine-grained than just the website itself could tell us, and
|
|
|
|
352
|
|
00:25:29,880 --> 00:25:36,240
|
|
I was already beginning to become interested in analytics data, and through a bit of serendipity
|
|
|
|
353
|
|
00:25:36,240 --> 00:25:41,400
|
|
came across Ronan, and we began working together on this project.
|
|
|
|
354
|
|
00:25:41,400 --> 00:25:47,520
|
|
So over to Ronan to explain a bit about how you went about this, and I'm going to advance
|
|
|
|
355
|
|
00:25:47,520 --> 00:25:50,760
|
|
the slides for you, Ronan, so just tell me, I'll go to the first one, then tell me when
|
|
|
|
356
|
|
00:25:50,760 --> 00:25:53,400
|
|
to move along.
|
|
|
|
357
|
|
00:25:53,400 --> 00:25:54,400
|
|
That's perfect.
|
|
|
|
358
|
|
00:25:54,400 --> 00:26:00,880
|
|
Thank you very much, Cathy, for preparing all this work.
|
|
|
|
359
|
|
00:26:00,880 --> 00:26:05,800
|
|
So yeah, there are many, many things to say here.
|
|
|
|
360
|
|
00:26:05,800 --> 00:26:12,840
|
|
I think the first thing to mention is that this has been, to me, really a project that's
|
|
|
|
361
|
|
00:26:12,840 --> 00:26:19,080
|
|
really motivated me a lot because it started by training, so the team of Cathy asked me
|
|
|
|
362
|
|
00:26:19,080 --> 00:26:27,240
|
|
to come in the UK and come and train our full team, so I took my backpack, I fly from France
|
|
|
|
363
|
|
00:26:27,240 --> 00:26:31,280
|
|
to the UK, and I trained our team for about three days.
|
|
|
|
364
|
|
00:26:31,280 --> 00:26:37,080
|
|
I think it was three, I don't remember if it was three or five days, but it's the kind
|
|
|
|
365
|
|
00:26:37,080 --> 00:26:43,000
|
|
of project which is really transforming you as an analyst because it's a project which
|
|
|
|
366
|
|
00:26:43,000 --> 00:26:45,440
|
|
really doesn't look like any others.
|
|
|
|
367
|
|
00:26:45,440 --> 00:26:50,080
|
|
It's not like a public website on which you can land on, and you can easily guess and
|
|
|
|
368
|
|
00:26:50,080 --> 00:26:54,200
|
|
find what are the different data collection points that you need to implement.
|
|
|
|
369
|
|
00:26:54,200 --> 00:27:00,080
|
|
Here in the case of the RAP project, the first thing I learned about is that it was using
|
|
|
|
370
|
|
00:27:00,080 --> 00:27:05,520
|
|
a PHP framework that I didn't know, which was Codignitor.
|
|
|
|
371
|
|
00:27:05,520 --> 00:27:13,960
|
|
The developer of the website is a third-party agency based in India, so it wasn't like I
|
|
|
|
372
|
|
00:27:13,960 --> 00:27:18,840
|
|
was talking to Cathy straight away, and Cathy could implement the different tracking code
|
|
|
|
373
|
|
00:27:18,840 --> 00:27:19,840
|
|
that I wanted.
|
|
|
|
374
|
|
00:27:19,840 --> 00:27:28,360
|
|
It was really like a three-party project, so the technology is named Codignitor.
|
|
|
|
375
|
|
00:27:28,360 --> 00:27:32,960
|
|
I wasn't really scared about the technology Codignitor that I didn't know, I just went
|
|
|
|
376
|
|
00:27:32,960 --> 00:27:36,840
|
|
to the famous search engine and found out that it was a PHP framework.
|
|
|
|
377
|
|
00:27:36,840 --> 00:27:41,960
|
|
From this, I knew already that it means a project that I could not put myself, my hands
|
|
|
|
378
|
|
00:27:41,960 --> 00:27:50,640
|
|
on, but that I would have to give a recommendation to a third-party company, so to set up the
|
|
|
|
379
|
|
00:27:50,640 --> 00:27:56,320
|
|
dev company, which means for me project management in terms of analytics projects.
|
|
|
|
380
|
|
00:27:56,320 --> 00:28:00,920
|
|
It means that I really needed to well structure my project in order for them to know what
|
|
|
|
381
|
|
00:28:00,920 --> 00:28:08,840
|
|
they need to implement, and I needed to ensure as well that Cathy and her team could clearly
|
|
|
|
382
|
|
00:28:08,840 --> 00:28:14,040
|
|
understand what we're going to implement, and that it aligns with everything that Cathy
|
|
|
|
383
|
|
00:28:14,040 --> 00:28:19,120
|
|
has mentioned before, which are the needs that they have in terms of data collection
|
|
|
|
384
|
|
00:28:19,120 --> 00:28:22,760
|
|
for this research project.
|
|
|
|
385
|
|
00:28:22,760 --> 00:28:27,800
|
|
The key aspect of the project, as Cathy showed to you, is a few pages.
|
|
|
|
386
|
|
00:28:27,800 --> 00:28:34,520
|
|
I think it's a maximum of between 15 to 20 pages, so it seems like that's an easy project,
|
|
|
|
387
|
|
00:28:34,520 --> 00:28:39,720
|
|
but with a lot in terms of data collection.
|
|
|
|
388
|
|
00:28:39,720 --> 00:28:46,200
|
|
You have a lot of advanced tracking code, which were necessary, including the measurement
|
|
|
|
389
|
|
00:28:46,200 --> 00:28:56,000
|
|
of events, including the use of custom dimensions to say level, visit level information data,
|
|
|
|
390
|
|
00:28:56,000 --> 00:29:04,240
|
|
and as well, we quickly saw it user ID, so the possibility to know who is the individual,
|
|
|
|
391
|
|
00:29:04,240 --> 00:29:09,360
|
|
let's say, who is making those different choices.
|
|
|
|
392
|
|
00:29:09,360 --> 00:29:15,320
|
|
We make the choices to go for a data layer, so to use Matomo Tag Manager, because the
|
|
|
|
393
|
|
00:29:15,320 --> 00:29:22,200
|
|
idea was to have something which was really consistent, because the thing is that we have
|
|
|
|
394
|
|
00:29:22,200 --> 00:29:29,520
|
|
not many pages, but all those pages are critical, and if we were going without a data layer,
|
|
|
|
395
|
|
00:29:29,520 --> 00:29:36,080
|
|
so to say, if I was taking in the project by just asking to add the container on all
|
|
|
|
396
|
|
00:29:36,080 --> 00:29:43,040
|
|
the pages, and then decide to use by either scrapping or either use automatically, let's
|
|
|
|
397
|
|
00:29:43,040 --> 00:29:49,480
|
|
say, the Tag Manager to collect the different data points, I chances that in the meantime,
|
|
|
|
398
|
|
00:29:49,480 --> 00:29:53,640
|
|
the dev company would have made changes to the DOM of the page, and everything would
|
|
|
|
399
|
|
00:29:53,640 --> 00:29:57,320
|
|
have exploded, and I would have to redo the full data collection.
|
|
|
|
400
|
|
00:29:57,320 --> 00:30:02,840
|
|
So that's the reason why we designed a really consistent data layer, which had been implemented
|
|
|
|
401
|
|
00:30:02,840 --> 00:30:08,000
|
|
directly by the dev company in order to ensure that if something breaks in terms of data
|
|
|
|
402
|
|
00:30:08,000 --> 00:30:13,320
|
|
collection, I wasn't the person responsible of screwing up the data collection, and of
|
|
|
|
403
|
|
00:30:13,320 --> 00:30:23,260
|
|
course, it gave more responsibility and involved more the dev team, which was really the right
|
|
|
|
404
|
|
00:30:23,260 --> 00:30:28,560
|
|
solution in this specific project, because I couldn't have the end on the source code.
|
|
|
|
405
|
|
00:30:28,560 --> 00:30:36,640
|
|
So those points are really critical, and this is more like a project management analytic
|
|
|
|
406
|
|
00:30:36,640 --> 00:30:46,000
|
|
system rather than, let's say, an analyst doing all the work on the platform.
|
|
|
|
407
|
|
00:30:46,000 --> 00:30:55,400
|
|
We really need, as well, to have a clear quality assessment process, because as Katie shows,
|
|
|
|
408
|
|
00:30:55,400 --> 00:31:01,160
|
|
the RAP project is a place where people are ordering, so they're not making some purchase
|
|
|
|
409
|
|
00:31:01,160 --> 00:31:09,000
|
|
because the products are for free, but they are making some orders, and once you make
|
|
|
|
410
|
|
00:31:09,000 --> 00:31:15,880
|
|
an order, you cannot order back the different elements, so it means that we didn't want
|
|
|
|
411
|
|
00:31:15,880 --> 00:31:21,840
|
|
all the time to go to Katie and say, okay, we make some order in order to test our data
|
|
|
|
412
|
|
00:31:21,840 --> 00:31:27,280
|
|
collection, please, could you remove us from the system, and we have to make another order
|
|
|
|
413
|
|
00:31:27,280 --> 00:31:32,920
|
|
again in order to test the platform, so that's why we needed to really have a clear project
|
|
|
|
414
|
|
00:31:32,920 --> 00:31:38,440
|
|
management system in which we can ensure, okay, here the tracking code has been implemented
|
|
|
|
415
|
|
00:31:38,440 --> 00:31:42,920
|
|
over here, we test it, it works, okay, so we tick in the box, it doesn't work, and then
|
|
|
|
416
|
|
00:31:42,920 --> 00:31:49,240
|
|
we go back to the dev team in order to ask for a new implementation, and the other point
|
|
|
|
417
|
|
00:31:49,240 --> 00:31:56,240
|
|
which was of critical importance was the use of custom dimension, so to say to add additional
|
|
|
|
418
|
|
00:31:56,240 --> 00:32:02,600
|
|
data which were not collected by default within Matomo to each individual, so as the use of
|
|
|
|
419
|
|
00:32:02,600 --> 00:32:09,440
|
|
the premium feature named custom report, which was one of critical importance because as
|
|
|
|
420
|
|
00:32:09,440 --> 00:32:17,800
|
|
we just saw it, the need of data is big, and it has to be crossed with different dimensions,
|
|
|
|
421
|
|
00:32:17,800 --> 00:32:23,280
|
|
so this is where the use of custom dimension was of critical importance.
|
|
|
|
422
|
|
00:32:23,280 --> 00:32:31,920
|
|
I'm done for this slide, okay, so next one is the work that we decided to work, so this
|
|
|
|
423
|
|
00:32:31,920 --> 00:32:38,280
|
|
is the project management part, this work has been reviewed by a colleague of mine that
|
|
|
|
424
|
|
00:32:38,280 --> 00:32:44,000
|
|
you know who made a conference as well named Frédéric Forster, and this is the way we
|
|
|
|
425
|
|
00:32:44,000 --> 00:32:53,040
|
|
work with the agency, so it consisted of myself drafting this document reviewed by my colleague
|
|
|
|
426
|
|
00:32:53,040 --> 00:33:00,120
|
|
Frédéric in order to have a document which is listing what the data layer is about and
|
|
|
|
427
|
|
00:33:00,120 --> 00:33:07,080
|
|
then showing this document to the team of Cathy to explain to the team of Cathy what
|
|
|
|
428
|
|
00:33:07,080 --> 00:33:10,680
|
|
this is all about because that's clearly not the kind of thing that you see within the
|
|
|
|
429
|
|
00:33:10,680 --> 00:33:15,640
|
|
Matomo's documentation, here we are more looking at the project management document made by
|
|
|
|
430
|
|
00:33:15,640 --> 00:33:23,240
|
|
an analyst and which as well serve in terms of transparency with the techie team for them
|
|
|
|
431
|
|
00:33:23,240 --> 00:33:31,480
|
|
in order to see if, okay, this is the engagement that we took together, you add the responsibility
|
|
|
|
432
|
|
00:33:31,480 --> 00:33:38,640
|
|
of deploying the code which is on the column number let's say five for you which is named
|
|
|
|
433
|
|
00:33:38,640 --> 00:33:44,240
|
|
code to push when the action is made, here as you can see you can clearly see the data
|
|
|
|
434
|
|
00:33:44,240 --> 00:33:50,520
|
|
layer which was implemented, so in our case it was for every step of the order made by
|
|
|
|
435
|
|
00:33:50,520 --> 00:33:58,800
|
|
the user we could push an event to Matomo and as I previously said we add for this tracking
|
|
|
|
436
|
|
00:33:58,800 --> 00:34:04,720
|
|
code to be consistent so that's why we decided to use a data layer and in the last column
|
|
|
|
437
|
|
00:34:04,720 --> 00:34:09,080
|
|
of this document you can clearly see that there is our recommendation which is in our
|
|
|
|
438
|
|
00:34:09,080 --> 00:34:16,960
|
|
case that this data layer wasn't implemented properly so please review on this given page
|
|
|
|
439
|
|
00:34:16,960 --> 00:34:22,160
|
|
that you have well included this given piece of code on the button when someone is pressing
|
|
|
|
440
|
|
00:34:22,160 --> 00:34:28,360
|
|
it so really here to understand how to deal with such a big project it's really like project
|
|
|
|
441
|
|
00:34:28,360 --> 00:34:33,560
|
|
management before all that that's the most that's the most important they have to precise
|
|
|
|
442
|
|
00:34:33,560 --> 00:34:39,640
|
|
as well one thing that we make some choices with with Katy and her team not to go for
|
|
|
|
443
|
|
00:34:39,640 --> 00:34:44,000
|
|
the e-commerce tracking code so just to let you know as well that when you deal with such
|
|
|
|
444
|
|
00:34:44,000 --> 00:34:50,360
|
|
a project you have some choices to make because as we saw the user can make some orders but
|
|
|
|
445
|
|
00:34:50,360 --> 00:34:56,160
|
|
those order are not paid orders so we had the choice to go either for e-commerce tracking
|
|
|
|
446
|
|
00:34:56,160 --> 00:35:00,840
|
|
code or either to go for the event tracking code and we decided to go for the event one
|
|
|
|
447
|
|
00:35:00,840 --> 00:35:07,280
|
|
because it was like more justified to go for it rather than having to create some additional
|
|
|
|
448
|
|
00:35:07,280 --> 00:35:14,360
|
|
data who were not existing within the system such as let's say the the tax amount such
|
|
|
|
449
|
|
00:35:14,360 --> 00:35:22,920
|
|
as the order price such as the transactional order that kind of thing I'm good for this
|
|
|
|
450
|
|
00:35:22,920 --> 00:35:34,920
|
|
part next one is kind of the similar thing the big difference is that on the previous
|
|
|
|
451
|
|
00:35:34,920 --> 00:35:41,960
|
|
slides that I was that we are showing it was about some variables that we are inserting
|
|
|
|
452
|
|
00:35:41,960 --> 00:35:47,040
|
|
within the data layer on this one this is another method that we use with the tag manager
|
|
|
|
453
|
|
00:35:47,040 --> 00:35:52,720
|
|
which is the push event method which consists of sending custom event to the data layer
|
|
|
|
454
|
|
00:35:52,720 --> 00:35:59,040
|
|
in order for it to react on the interaction of the of the visitor and as well this is
|
|
|
|
455
|
|
00:35:59,040 --> 00:36:10,120
|
|
how we succeed to handle it okay I'm good for this one okay and last but not least this
|
|
|
|
456
|
|
00:36:10,120 --> 00:36:17,440
|
|
slide show in fact all the different variables so those are all the different needs that
|
|
|
|
457
|
|
00:36:17,440 --> 00:36:26,080
|
|
we list with with Katy and ask for validation to see if there was any data which were missing
|
|
|
|
458
|
|
00:36:26,080 --> 00:36:33,160
|
|
out of this list and once validated of course this is this is what we implemented but here
|
|
|
|
459
|
|
00:36:33,160 --> 00:36:37,800
|
|
on small what we are showing is is really the project management part is like there
|
|
|
|
460
|
|
00:36:37,800 --> 00:36:42,400
|
|
is a lot of needs from the client but we need at some point to put a limit and say okay
|
|
|
|
461
|
|
00:36:42,400 --> 00:36:50,760
|
|
we don't need extra more data do we agree that this list is the limit and you all need
|
|
|
|
462
|
|
00:36:50,760 --> 00:36:57,720
|
|
those and if yes of course this is kind of what is finally defining the final price that's
|
|
|
|
463
|
|
00:36:57,720 --> 00:37:03,920
|
|
the analyst will put on on the quote let's say and as well yes I'm good with this slide
|
|
|
|
464
|
|
00:37:03,920 --> 00:37:11,720
|
|
I think that was the last oh yeah oh and that's as well a critical part of the of the project
|
|
|
|
465
|
|
00:37:11,720 --> 00:37:19,040
|
|
and that I really loved in it is that I during the training I showed to Katy and her team
|
|
|
|
466
|
|
00:37:19,040 --> 00:37:26,480
|
|
how to deal with the custom reports feature of matmo and I've been surprised to see that's
|
|
|
|
467
|
|
00:37:26,480 --> 00:37:32,320
|
|
when I came some days after or weeks after the implementation with the wrap project that
|
|
|
|
468
|
|
00:37:32,320 --> 00:37:38,000
|
|
a lot of custom reports were created and that's a good sign of the training works and now
|
|
|
|
469
|
|
00:37:38,000 --> 00:37:43,240
|
|
people have the solution well in end and they are creating as many custom reports as they
|
|
|
|
470
|
|
00:37:43,240 --> 00:37:51,880
|
|
can and here you can clearly see how the custom reports really enhance the data which were
|
|
|
|
471
|
|
00:37:51,880 --> 00:37:57,200
|
|
in the system of matmo because here we got the user ID with the different components
|
|
|
|
472
|
|
00:37:57,200 --> 00:38:05,880
|
|
that it shows and the date at which the individual decided to pick up the product I think I have
|
|
|
|
473
|
|
00:38:05,880 --> 00:38:13,440
|
|
one slide left yes and here is another one another report which is really showing for
|
|
|
|
474
|
|
00:38:13,440 --> 00:38:18,440
|
|
each individual what are the different components and you can clearly see that on line number
|
|
|
|
475
|
|
00:38:18,440 --> 00:38:25,880
|
|
one from line number one to line number four the individual decided to pick up all the
|
|
|
|
476
|
|
00:38:25,880 --> 00:38:31,480
|
|
components but line number five for some reason this individual decided to not pick up all
|
|
|
|
477
|
|
00:38:31,480 --> 00:38:36,880
|
|
the different components which was one of the needs which was requested from this research
|
|
|
|
478
|
|
00:38:36,880 --> 00:38:43,600
|
|
which is about okay what are the different choices that people are making and I think
|
|
|
|
479
|
|
00:38:43,600 --> 00:38:51,800
|
|
I'm good for my slides yep exactly okay thanks Ronan so in the last section before the Q&A
|
|
|
|
480
|
|
00:38:51,800 --> 00:38:57,260
|
|
I'll just present some early insights from our data collection so from the custom reports
|
|
|
|
481
|
|
00:38:57,260 --> 00:39:01,720
|
|
that Ronan was just talking about we've been able to learn a lot already we haven't finished
|
|
|
|
482
|
|
00:39:01,720 --> 00:39:07,160
|
|
collecting our data yet we're just a few months into the project so we haven't learned everything
|
|
|
|
483
|
|
00:39:07,160 --> 00:39:13,240
|
|
that we will be able to learn by the end but I'll show you what we've got so far so this
|
|
|
|
484
|
|
00:39:13,240 --> 00:39:20,200
|
|
is just how I use the data and what's working best for me is to think about the flow of
|
|
|
|
485
|
|
00:39:20,200 --> 00:39:24,640
|
|
people through the website in order to sort of see where the things going wrong within
|
|
|
|
486
|
|
00:39:24,640 --> 00:39:30,040
|
|
the website and if something needs kind of attending to before our full trial so things
|
|
|
|
487
|
|
00:39:30,040 --> 00:39:34,800
|
|
that might be leading people to drop out basically so the non-use usage attrition that I talked
|
|
|
|
488
|
|
00:39:34,800 --> 00:39:43,880
|
|
about before so 91 people at the point at which I downloaded the data from the custom
|
|
|
|
489
|
|
00:39:43,880 --> 00:39:48,680
|
|
reports had at that point been randomized to wrap so gone to that top of the two different
|
|
|
|
490
|
|
00:39:48,680 --> 00:39:56,680
|
|
arms in the randomization and of those 56 people clicked on the link so I told you about
|
|
|
|
491
|
|
00:39:56,680 --> 00:40:03,520
|
|
how our software RedCat sent an email invite to everybody inviting them to go to one of
|
|
|
|
492
|
|
00:40:03,520 --> 00:40:09,440
|
|
the two different websites so this is 91 people sent the invite to wrapped and 56 of them
|
|
|
|
493
|
|
00:40:09,440 --> 00:40:13,080
|
|
clicked on the link and went through to the website so not everybody's doing that despite
|
|
|
|
494
|
|
00:40:13,080 --> 00:40:21,960
|
|
us incentivizing them with a voucher so we've lost 35 at that point and then we when I first
|
|
|
|
495
|
|
00:40:21,960 --> 00:40:31,400
|
|
land on the website they have to create a profile and sort of register so and then they
|
|
|
|
496
|
|
00:40:31,400 --> 00:40:36,560
|
|
also complete some tailoring questions so these are the ones that determine how many
|
|
|
|
497
|
|
00:40:36,560 --> 00:40:41,880
|
|
of the components of the six components that they get so between them sort of landing on
|
|
|
|
498
|
|
00:40:41,880 --> 00:40:45,880
|
|
the website and creating a profile we've lost just two people in that stage of completing
|
|
|
|
499
|
|
00:40:45,880 --> 00:40:50,120
|
|
the tailoring questions so I was a little bit concerned I think there's 10 questions
|
|
|
|
500
|
|
00:40:50,120 --> 00:40:55,640
|
|
in total at me people might get a little irritated by that sort of section but we've only lost
|
|
|
|
501
|
|
00:40:55,640 --> 00:41:01,680
|
|
two there so that's quite a good sign for me and then we've got registration being complete
|
|
|
|
502
|
|
00:41:01,680 --> 00:41:09,840
|
|
and then between that stage and oh sorry visiting the home page we've lost nobody so that's
|
|
|
|
503
|
|
00:41:09,840 --> 00:41:13,800
|
|
another good sign that that whole stage is working well so our biggest concern really
|
|
|
|
504
|
|
00:41:13,800 --> 00:41:19,120
|
|
is that email that's encouraging people to go visit in the first place perhaps is a bit
|
|
|
|
505
|
|
00:41:19,120 --> 00:41:24,240
|
|
boring and doesn't sell the website very well although in a trial context it's kind of difficult
|
|
|
|
506
|
|
00:41:24,240 --> 00:41:27,080
|
|
to do that because we can't spell out the things that are good about the intervention
|
|
|
|
507
|
|
00:41:27,080 --> 00:41:35,720
|
|
website very easily in that context but we could perhaps try harder and then next slide
|
|
|
|
508
|
|
00:41:35,720 --> 00:41:42,880
|
|
so Ronan was talking about people being assigned different components and I mentioned how there's
|
|
|
|
509
|
|
00:41:42,880 --> 00:41:48,120
|
|
up to six different ones so we were interested in knowing about the proportion of people
|
|
|
|
510
|
|
00:41:48,120 --> 00:41:51,960
|
|
that were going to get assigned the different numbers of them because when we started out
|
|
|
|
511
|
|
00:41:51,960 --> 00:41:56,520
|
|
they said we really didn't know that so they answer these 10 questions at the start of
|
|
|
|
512
|
|
00:41:56,520 --> 00:42:01,040
|
|
the website and depending on their answers they'll get assigned either all of the six
|
|
|
|
513
|
|
00:42:01,040 --> 00:42:06,160
|
|
of them or just the one of them there's always a minimum of one and this is how it's worked
|
|
|
|
514
|
|
00:42:06,160 --> 00:42:10,360
|
|
out so no one's had just one even though that's a possibility and then very few people have
|
|
|
|
515
|
|
00:42:10,360 --> 00:42:18,720
|
|
had two three or four components most people have either been assigned five or six components
|
|
|
|
516
|
|
00:42:18,720 --> 00:42:22,040
|
|
so that's useful in terms of helping us plan for the main trial it helps us to work out
|
|
|
|
517
|
|
00:42:22,040 --> 00:42:25,520
|
|
resource need because some of the components at least where they're ordering something
|
|
|
|
518
|
|
00:42:25,520 --> 00:42:34,800
|
|
has a cost associated that's really useful and then I explained that there were three
|
|
|
|
519
|
|
00:42:34,800 --> 00:42:38,240
|
|
different things that people could order and I showed you the sample box that the website
|
|
|
|
520
|
|
00:42:38,240 --> 00:42:43,160
|
|
page relating to that so this is the box with 12 different types of condoms to try out in
|
|
|
|
521
|
|
00:42:43,160 --> 00:42:49,920
|
|
it so 54 people were assigned the sample box in total that's everyone who went to the website
|
|
|
|
522
|
|
00:42:49,920 --> 00:42:55,320
|
|
because everyone gets the sample box by default so 54 were given that and then so on that
|
|
|
|
523
|
|
00:42:55,320 --> 00:43:00,040
|
|
main page where you've got the six different boxes that you can click on or to take you
|
|
|
|
524
|
|
00:43:00,040 --> 00:43:05,200
|
|
through to the relevant page 54 people have seen the one for the sample box and 48 of
|
|
|
|
525
|
|
00:43:05,200 --> 00:43:09,080
|
|
them have actually clicked on that and gone to visit that page we've just lost six at
|
|
|
|
526
|
|
00:43:09,080 --> 00:43:15,800
|
|
that point which is not bad and then between visiting that page and then placing an order
|
|
|
|
527
|
|
00:43:15,800 --> 00:43:20,400
|
|
we've lost 11 people so that's 11 people maybe who just didn't like the look of it and just
|
|
|
|
528
|
|
00:43:20,400 --> 00:43:24,440
|
|
thought that wasn't really for them but that's an assumption but something perhaps to speak
|
|
|
|
529
|
|
00:43:24,440 --> 00:43:29,800
|
|
to people later on in the study when we come to our interviews and then those who then
|
|
|
|
530
|
|
00:43:29,800 --> 00:43:33,160
|
|
clicked to go through to the next page they're sort of progressing with making an order they've
|
|
|
|
531
|
|
00:43:33,160 --> 00:43:37,320
|
|
chosen what box they want what insert they want we've just lost one person and then we
|
|
|
|
532
|
|
00:43:37,320 --> 00:43:42,360
|
|
lost two on the next page between the point to seeing that second page and actually placing
|
|
|
|
533
|
|
00:43:42,360 --> 00:43:47,720
|
|
the order so yeah I was quite pleased that it shows to me that process of ordering is
|
|
|
|
534
|
|
00:43:47,720 --> 00:43:51,960
|
|
quite smooth and straightforward and perhaps not no one's experiencing technical problems
|
|
|
|
535
|
|
00:43:51,960 --> 00:43:58,000
|
|
or issues with usability around the ordering process and really it's the same picture with
|
|
|
|
536
|
|
00:43:58,000 --> 00:44:02,920
|
|
condom ordering so 49 people ordered that I'm not going to go through all the numbers
|
|
|
|
537
|
|
00:44:02,920 --> 00:44:07,600
|
|
on this one but it's the same pattern that we've lost just a handful at the start who
|
|
|
|
538
|
|
00:44:07,600 --> 00:44:12,760
|
|
were never visiting that page and then the actual ordering seems to be fairly straightforward
|
|
|
|
539
|
|
00:44:12,760 --> 00:44:20,960
|
|
for everybody and the same with ordering the condom carrier as well and then the three
|
|
|
|
540
|
|
00:44:20,960 --> 00:44:24,240
|
|
different types of video that people can go and see I'm just going to do one slide on
|
|
|
|
541
|
|
00:44:24,240 --> 00:44:27,800
|
|
one of the videos which is the demo demo video that I showed you it's a completely different
|
|
|
|
542
|
|
00:44:27,800 --> 00:44:35,440
|
|
picture here so 47 people allocated this video but only 16 actually clicked on that box to
|
|
|
|
543
|
|
00:44:35,440 --> 00:44:40,400
|
|
go and visit the demo page that's telling us that that home page is not selling this
|
|
|
|
544
|
|
00:44:40,400 --> 00:44:47,600
|
|
demo video at all goodbye even going to click on it well only 16 of our 47 did and then
|
|
|
|
545
|
|
00:44:47,600 --> 00:44:53,080
|
|
only of those 16 over only 7 actually then went ahead to click on the video and play
|
|
|
|
546
|
|
00:44:53,080 --> 00:44:58,440
|
|
it and then nobody watched it in full and the meantime is about watching the video was
|
|
|
|
547
|
|
00:44:58,440 --> 00:45:03,920
|
|
17 seconds so not attractive able to go and visit it when they do go and visit the page
|
|
|
|
548
|
|
00:45:03,920 --> 00:45:09,040
|
|
not playing it very much not many people playing it and then when they do disengaging quite
|
|
|
|
549
|
|
00:45:09,040 --> 00:45:14,080
|
|
quickly so we've got quite a bit of work to do on the video and the demo video but the
|
|
|
|
550
|
|
00:45:14,080 --> 00:45:21,320
|
|
same picture is true of the other types of videos as well they all need looking at so
|
|
|
|
551
|
|
00:45:21,320 --> 00:45:26,320
|
|
what we've learnt well we've learnt that we can use analytics data to gain some important
|
|
|
|
552
|
|
00:45:26,320 --> 00:45:30,680
|
|
insights into non usage attrition and we've just started to kind of touch the surface
|
|
|
|
553
|
|
00:45:30,680 --> 00:45:34,520
|
|
with the data we've got already and we're going to use this to make some improvements
|
|
|
|
554
|
|
00:45:34,520 --> 00:45:41,400
|
|
so my conclusions I've kind of touched on touched upon already are that the home page
|
|
|
|
555
|
|
00:45:41,400 --> 00:45:45,480
|
|
as attracting users to visit are the first three components the things that you can order
|
|
|
|
556
|
|
00:45:45,480 --> 00:45:50,680
|
|
and once on those component pages there's good conversion to ordering there seems to
|
|
|
|
557
|
|
00:45:50,680 --> 00:45:55,400
|
|
be no issues with the ordering process but the home page is not attracting people to
|
|
|
|
558
|
|
00:45:55,400 --> 00:46:03,000
|
|
visit the video content and those video component pages are also not working very well to encourage
|
|
|
|
559
|
|
00:46:03,000 --> 00:46:05,920
|
|
users to watch the videos when they're already there and then the videos aren't being watched
|
|
|
|
560
|
|
00:46:05,920 --> 00:46:11,400
|
|
for very long so the first few seconds are encouraging people to carry on and we still
|
|
|
|
561
|
|
00:46:11,400 --> 00:46:15,960
|
|
have lots and lots to learn about this field and what is really possible in terms of analytics
|
|
|
|
562
|
|
00:46:15,960 --> 00:46:28,080
|
|
so I'm very much a newbie in this situation and our solution for linking individual level
|
|
|
|
563
|
|
00:46:28,080 --> 00:46:36,960
|
|
the user ID that we're collecting in redcap for all our surveys and STI testing and linking
|
|
|
|
564
|
|
00:46:36,960 --> 00:46:41,520
|
|
that to our individuals as they go through to the two different websites so we can collect
|
|
|
|
565
|
|
00:46:41,520 --> 00:46:47,240
|
|
this analytics data that process that Ronan worked out for us by liaising with us and
|
|
|
|
566
|
|
00:46:47,240 --> 00:46:51,680
|
|
also our development companies has worked really well has been successful had no problems
|
|
|
|
567
|
|
00:46:51,680 --> 00:46:57,480
|
|
with gathering the unique user ID of everyone in Amatomo and seeing what they're all doing
|
|
|
|
568
|
|
00:46:57,480 --> 00:47:01,240
|
|
creating these custom reports and gathering all that data from there that's all worked
|
|
|
|
569
|
|
00:47:01,240 --> 00:47:05,120
|
|
really well and that means it's going to be possible for us to measure those in our main
|
|
|
|
570
|
|
00:47:05,120 --> 00:47:10,800
|
|
trial and in quite a fine-grained way which is really useful and we're just beginning
|
|
|
|
571
|
|
00:47:10,800 --> 00:47:15,520
|
|
to think about how we'll calculate a sort of standardized score for everybody in our
|
|
|
|
572
|
|
00:47:15,520 --> 00:47:20,760
|
|
trial at an individual level based on the products that are allocated and ordered and
|
|
|
|
573
|
|
00:47:20,760 --> 00:47:30,280
|
|
also which videos to watch and how long for so that's the end of the presentation so I'd
|
|
|
|
574
|
|
00:47:30,280 --> 00:47:34,160
|
|
just like to acknowledge our funder the National Institute for Health Research and all of the
|
|
|
|
575
|
|
00:47:34,160 --> 00:47:38,920
|
|
team working on this with me as well at the University of Hertfordshire and others and
|
|
|
|
576
|
|
00:47:38,920 --> 00:47:47,840
|
|
yeah on to any questions thank you very much Katie thank you very much for this great presentation
|
|
|
|
577
|
|
00:47:47,840 --> 00:47:55,240
|
|
I'm looking at the chat right now we have approximately let's say maximum five minutes
|
|
|
|
578
|
|
00:47:55,240 --> 00:48:00,440
|
|
because after there's a other talker at 4 p.m. I'm just going to pick the first one which
|
|
|
|
579
|
|
00:48:00,440 --> 00:48:08,200
|
|
is which was the most useful metamode tools used during this project?
|
|
|
|
580
|
|
00:48:08,200 --> 00:48:15,040
|
|
Rony you might have a view on this but for me I suppose I might not be using the right
|
|
|
|
581
|
|
00:48:15,040 --> 00:48:22,400
|
|
terms but it was the custom ID and then also the reports as well I mean the reports are
|
|
|
|
582
|
|
00:48:22,400 --> 00:48:28,160
|
|
something I go into all the time now to look at to gather the data to help me understand
|
|
|
|
583
|
|
00:48:28,160 --> 00:48:33,240
|
|
the usability of the website and on all the slides I've just presented with those flowcharts
|
|
|
|
584
|
|
00:48:33,240 --> 00:48:38,280
|
|
all comes from those custom reports.
|
|
|
|
585
|
|
00:48:38,280 --> 00:48:43,240
|
|
I'm looking at the chat there's another question with small linking with logistic which is
|
|
|
|
586
|
|
00:48:43,240 --> 00:48:48,760
|
|
about will it be possible to download the presentation I mean will it be possible for
|
|
|
|
587
|
|
00:48:48,760 --> 00:48:54,640
|
|
you to send it to I mean do you allow me to share it with the audience or would you like
|
|
|
|
588
|
|
00:48:54,640 --> 00:48:58,680
|
|
to keep your presentation for yourself or can it be?
|
|
|
|
589
|
|
00:48:58,680 --> 00:49:05,080
|
|
That's fine Rony yeah I'm happy for anyone to have it so I'll send it to you.
|
|
|
|
590
|
|
00:49:05,080 --> 00:49:11,440
|
|
Yeah well I think I have it in my mailbox so I will just make it as a PDF and then and
|
|
|
|
591
|
|
00:49:11,440 --> 00:49:15,400
|
|
then share it back.
|
|
|
|
592
|
|
00:49:15,400 --> 00:49:23,320
|
|
I let the audience write down the last question we still have let's say four minutes left.
|
|
|
|
593
|
|
00:49:23,320 --> 00:49:28,160
|
|
Do you have for example any questions Cathy that you would like me to ask you for the
|
|
|
|
594
|
|
00:49:28,160 --> 00:49:35,400
|
|
audience or do you have any questions for you?
|
|
|
|
595
|
|
00:49:35,400 --> 00:49:39,720
|
|
I mean do you have any questions that you would have expected the audience to ask you
|
|
|
|
596
|
|
00:49:39,720 --> 00:49:43,840
|
|
and that you would like me to ask you?
|
|
|
|
597
|
|
00:49:43,840 --> 00:49:49,320
|
|
Do I have any questions?
|
|
|
|
598
|
|
00:49:49,320 --> 00:49:58,560
|
|
I suppose the thing that I'm still struggling with the most is linking all the data in the
|
|
|
|
599
|
|
00:49:58,560 --> 00:50:06,280
|
|
custom report so each custom report I can download the data as an excel file so for
|
|
|
|
600
|
|
00:50:06,280 --> 00:50:09,880
|
|
example if we're talking about the components allocated I can download it so I've got one
|
|
|
|
601
|
|
00:50:09,880 --> 00:50:14,920
|
|
column which is the user ID and then you know did they have component one yes no component
|
|
|
|
602
|
|
00:50:14,920 --> 00:50:20,320
|
|
two yes no I can create it like that then I can download another separate report that
|
|
|
|
603
|
|
00:50:20,320 --> 00:50:27,440
|
|
says did they visit the sample pack page for example and that'll be a completely separate
|
|
|
|
604
|
|
00:50:27,440 --> 00:50:33,440
|
|
one and so at the moment they're all very separate like that and then I have to combine
|
|
|
|
605
|
|
00:50:33,440 --> 00:50:43,840
|
|
them all sort of together so I suppose oh and also every time I want to update the data
|
|
|
|
606
|
|
00:50:43,840 --> 00:50:49,880
|
|
it's not easy I can use date I suppose but I can't just filter by everything since I
|
|
|
|
607
|
|
00:50:49,880 --> 00:50:53,880
|
|
last looked and then download that and add it in I haven't just kept kind of keep manually
|
|
|
|
608
|
|
00:50:53,880 --> 00:51:00,480
|
|
looking and then updating my excel data based on that so I don't know whether it's possible
|
|
|
|
609
|
|
00:51:00,480 --> 00:51:05,080
|
|
this is more a question for you than the audience but this is something that I'm I'm struggling
|
|
|
|
610
|
|
00:51:05,080 --> 00:51:12,000
|
|
most with at the moment and yeah I'd be keen to know about if there's a way to kind of
|
|
|
|
611
|
|
00:51:12,000 --> 00:51:17,280
|
|
make that process a bit more straightforward I suppose but might be getting to the limits
|
|
|
|
612
|
|
00:51:17,280 --> 00:51:25,160
|
|
of what Matomo can provide okay great yeah I don't have the yet the answer I think that
|
|
|
|
613
|
|
00:51:25,160 --> 00:51:32,760
|
|
we are looking for custom reports with far more dimensions I guess that's the key thing
|
|
|
|
614
|
|
00:51:32,760 --> 00:51:39,880
|
|
here the maximum is really three custom dimension and from what I remembered yeah every time
|
|
|
|
615
|
|
00:51:39,880 --> 00:51:46,040
|
|
you asked something I did my best in terms of optimizations you can yeah you can just
|
|
|
|
616
|
|
00:51:46,040 --> 00:51:51,480
|
|
put up to three custom dimension and then filter by some other custom dimension but
|
|
|
|
617
|
|
00:51:51,480 --> 00:51:56,240
|
|
at the moment I'm not sure that there's plans to extend the number of custom dimension which
|
|
|
|
618
|
|
00:51:56,240 --> 00:52:01,320
|
|
correspond more to what you would like like you extend if you export the full excel file
|
|
|
|
619
|
|
00:52:01,320 --> 00:52:05,000
|
|
and you will get as many custom dimension as you need so in our case I don't remember
|
|
|
|
620
|
|
00:52:05,000 --> 00:52:13,880
|
|
if we have up to five or ten but then you can refilter back in excel Alfonso says thank
|
|
|
|
621
|
|
00:52:13,880 --> 00:52:19,800
|
|
you because you answered his question about which was the most useful Matomo tools and
|
|
|
|
622
|
|
00:52:19,800 --> 00:52:28,320
|
|
we don't have any questions left so I guess that's the time to close the conference talk
|
|
|
|
623
|
|
00:52:28,320 --> 00:52:34,840
|
|
once more thank you very much Katy for your time thank you very much for the presentation
|
|
|
|
624
|
|
00:52:34,840 --> 00:52:39,640
|
|
and as well I would like to thank all the attendees of this conference and I wish you
|
|
|
|
625
|
|
00:52:39,640 --> 00:52:46,840
|
|
all a great end of Matomo camp there's just one conference two conferences left starting
|
|
|
|
626
|
|
00:52:46,840 --> 00:52:54,200
|
|
in five minutes and then we have the closing ceremony at in one hour so at five p.m. thank
|
|
|
|
627
|
|
00:52:54,200 --> 00:53:21,200
|
|
you very much pleasure good luck with the rest of the conference thanks bye bye
|
|
|