recording-subtitles/2021/Million Pageviews/Project.kdenlive.srt

1
00:00:00,270 --> 00:00:08,454
the everyone thank you for attending this

2
00:00:08,454 --> 00:00:10,857
session today we will have her her

3
00:00:10,857 --> 00:00:14,400
powers who has been working for that

4
00:00:14,400 --> 00:00:18,900
ten years with analytics seo development and

5
00:00:18,960 --> 00:00:19,740
devops

6
00:00:20,430 --> 00:00:22,020
he stopped the day is how to

7
00:00:22,020 --> 00:00:24,450
manage a moto mo with millions of

8
00:00:24,450 --> 00:00:27,690
page views per month where he will

9
00:00:27,690 --> 00:00:29,640
tell us more about their work at

10
00:00:29,700 --> 00:00:32,820
digital is sweden a company which matches

11
00:00:32,880 --> 00:00:35,520
manages some of the biggest matomo is

12
00:00:35,520 --> 00:00:38,580
the leaders in sweden so i'm giving

13
00:00:38,580 --> 00:00:39,900
the word to her now

14
00:00:42,930 --> 00:00:43,470
thank you

15
00:00:45,720 --> 00:00:46,950
arrest timer

16
00:00:58,530 --> 00:01:00,000
as you heard that i've been working

17
00:01:00,000 --> 00:01:02,190
with politics for many years

18
00:01:04,620 --> 00:01:08,310
i've been programming since i was a

19
00:01:08,310 --> 00:01:09,930
kid in the nineties

20
00:01:11,280 --> 00:01:16,590
i started professionally about eleven twelve years

21
00:01:16,590 --> 00:01:18,180
ago working there

22
00:01:19,800 --> 00:01:20,880
mostly as a

23
00:01:24,060 --> 00:01:26,880
from thunder from the beginning but yeah

24
00:01:27,090 --> 00:01:30,496
i wouldn't label mass front and back

25
00:01:30,510 --> 00:01:31,110
younger

26
00:01:32,160 --> 00:01:32,790
zero exp

27
00:01:33,840 --> 00:01:37,170
i am a generalist and am a

28
00:01:37,470 --> 00:01:39,750
wherever i needed so i'm a problem

29
00:01:39,750 --> 00:01:40,980
solver he could call me

30
00:01:42,990 --> 00:01:45,390
i have recently been on parental leave

31
00:01:45,390 --> 00:01:46,020
so

32
00:01:47,430 --> 00:01:50,250
i'm just back where cancel this presentation

33
00:01:50,250 --> 00:01:51,060
is sir

34
00:01:55,410 --> 00:01:59,370
put together recently so bear with me

35
00:02:00,150 --> 00:02:01,560
but i'm sure i can

36
00:02:03,348 --> 00:02:05,280
any questions if you have any

37
00:02:06,720 --> 00:02:08,990
yeah that's from above so i worked

38
00:02:08,990 --> 00:02:10,650
for digital lists now i've been here

39
00:02:10,650 --> 00:02:14,760
for six years and i've been working

40
00:02:14,760 --> 00:02:16,260
with the devops team

41
00:02:17,550 --> 00:02:18,300
i've been

42
00:02:20,130 --> 00:02:21,240
project manager

43
00:02:22,620 --> 00:02:23,940
and i have been working with moto

44
00:02:23,940 --> 00:02:24,420
more

45
00:02:25,440 --> 00:02:26,940
for a three or four years now

46
00:02:27,180 --> 00:02:28,380
in the digital lists

47
00:02:30,360 --> 00:02:31,830
so what what am i going to

48
00:02:31,830 --> 00:02:32,430
talk about

49
00:02:35,340 --> 00:02:37,680
when we started with matomo it was

50
00:02:37,680 --> 00:02:38,850
still called piwik

51
00:02:40,410 --> 00:02:45,120
and everyone knows that the gdp are

52
00:02:45,750 --> 00:02:47,790
in two thousand and eighteen was sarah

53
00:02:48,840 --> 00:02:54,330
getting air in effect so the gop

54
00:02:54,540 --> 00:02:57,390
were to recover from the seasons we

55
00:02:57,390 --> 00:02:58,050
number

56
00:03:00,270 --> 00:03:03,360
to attract data and take ownership or

57
00:03:04,440 --> 00:03:07,560
data so the swedish tax agency scott

58
00:03:07,560 --> 00:03:09,420
the rackets and

59
00:03:10,800 --> 00:03:13,710
they had a procurement and reached out

60
00:03:13,710 --> 00:03:15,510
to us and asked us if we

61
00:03:15,510 --> 00:03:19,079
knew matomo and we have had pv

62
00:03:19,079 --> 00:03:20,550
kit installations

63
00:03:21,690 --> 00:03:24,750
running for probably ten years so

64
00:03:26,040 --> 00:03:28,680
but we weren't no experts yet and

65
00:03:31,025 --> 00:03:33,840
always took on this said and

66
00:03:35,940 --> 00:03:37,440
yeah so a little bit

67
00:03:41,100 --> 00:03:43,350
sorry didn't it sir

68
00:03:45,750 --> 00:03:48,300
small country in europe if you look

69
00:03:48,300 --> 00:03:51,240
at the population but still we have

70
00:03:51,360 --> 00:03:54,865
ten million plus population in sweden and

71
00:03:56,580 --> 00:03:57,060
yeah

72
00:04:01,110 --> 00:04:04,320
the fiscal declaration i mean the sweden

73
00:04:04,320 --> 00:04:06,030
is very progressive when it comes to

74
00:04:06,060 --> 00:04:08,880
digital age digitalization and the

75
00:04:10,620 --> 00:04:13,770
in python twenty one four point five

76
00:04:13,770 --> 00:04:15,930
million people the their fiscal declaration on

77
00:04:15,930 --> 00:04:16,560
line and

78
00:04:18,209 --> 00:04:21,959
and all of those people went into

79
00:04:21,959 --> 00:04:27,210
this earth to say what the declaration

80
00:04:27,330 --> 00:04:29,340
and they have an utter or my

81
00:04:29,340 --> 00:04:32,910
services as well on other websites so

82
00:04:33,208 --> 00:04:37,554
lot of traffic to their sites and

83
00:04:40,590 --> 00:04:42,870
if you look at the timeline hair

84
00:04:42,900 --> 00:04:48,720
and you think how big applications were

85
00:04:48,720 --> 00:04:50,700
driven back in the days

86
00:04:51,930 --> 00:04:52,711
i

87
00:04:54,540 --> 00:04:56,340
i have it

88
00:04:56,520 --> 00:04:59,460
fire up or withdrawn on a server

89
00:05:01,265 --> 00:05:02,820
i checked our

90
00:05:05,070 --> 00:05:07,020
the kind of cirrhosis and then in

91
00:05:07,020 --> 00:05:09,060
two thousand and fourteen cabinet this came

92
00:05:09,360 --> 00:05:13,770
in and change how the landscape of

93
00:05:14,730 --> 00:05:17,370
applications are deployed today so

94
00:05:19,200 --> 00:05:21,240
and we signed with the captive etiquette

95
00:05:21,420 --> 00:05:23,700
just before two thousand and eighteen and

96
00:05:23,700 --> 00:05:26,130
so we had a bit time before

97
00:05:26,400 --> 00:05:28,770
may twenty fifth i think it was

98
00:05:28,860 --> 00:05:30,930
the date when gd pure was in

99
00:05:30,930 --> 00:05:31,470
effect

100
00:05:33,810 --> 00:05:36,510
and at this time if you downloaded

101
00:05:37,800 --> 00:05:40,440
the source code to pay vegas like

102
00:05:40,440 --> 00:05:42,000
it was called back in the day

103
00:05:43,440 --> 00:05:46,050
you would you could go to the

104
00:05:46,050 --> 00:05:48,090
online documentation and see that

105
00:05:49,230 --> 00:05:50,070
it's a

106
00:05:52,080 --> 00:05:54,176
a lamp stack basically so you have

107
00:05:54,176 --> 00:05:58,050
a leanness linux a part share and

108
00:05:58,710 --> 00:05:59,490
my sql

109
00:06:01,200 --> 00:06:02,190
and pitch p

110
00:06:04,167 --> 00:06:06,150
and for us this is a yeah

111
00:06:06,150 --> 00:06:08,220
we've been working with open source for

112
00:06:08,370 --> 00:06:12,900
twenty years or so so this is

113
00:06:12,900 --> 00:06:13,920
a very familiar

114
00:06:15,330 --> 00:06:15,870
for us

115
00:06:17,550 --> 00:06:18,780
but the challenge share

116
00:06:20,250 --> 00:06:23,100
guessing that cuban earth as was sir

117
00:06:23,190 --> 00:06:25,350
open source in two thousand and fourteen

118
00:06:26,580 --> 00:06:28,890
and then

119
00:06:31,710 --> 00:06:35,700
yeah it was clear how to a

120
00:06:35,760 --> 00:06:39,720
hostess a big installation where we knew

121
00:06:39,720 --> 00:06:41,190
that we were going to have a

122
00:06:41,190 --> 00:06:44,010
lot of traffic so that we put

123
00:06:44,010 --> 00:06:47,250
together a team of our engineers to

124
00:06:47,370 --> 00:06:50,160
try to figure out the best option

125
00:06:50,730 --> 00:06:51,480
to go for it

126
00:06:56,130 --> 00:06:59,010
yeah until two thousand and nineteen we

127
00:06:59,010 --> 00:07:00,180
could find

128
00:07:01,590 --> 00:07:05,370
more documentation on the motto more

129
00:07:06,600 --> 00:07:10,260
website at around the big insulation four

130
00:07:10,320 --> 00:07:12,030
hundred million page views per month or

131
00:07:12,030 --> 00:07:12,570
less

132
00:07:14,760 --> 00:07:15,480
but this is

133
00:07:16,740 --> 00:07:19,080
similar to the conclusion we came up

134
00:07:19,080 --> 00:07:20,010
with at the time

135
00:07:21,690 --> 00:07:22,410
i

136
00:07:25,050 --> 00:07:26,340
but when it comes to

137
00:07:28,200 --> 00:07:31,560
are running this and monitoring in the

138
00:07:31,564 --> 00:07:34,351
us on this silly challenge as we

139
00:07:34,351 --> 00:07:36,660
can just play some servers and hopefully

140
00:07:36,780 --> 00:07:39,180
they will run for a long time

141
00:07:39,180 --> 00:07:39,510
so

142
00:07:40,680 --> 00:07:42,150
we started to sketch our own

143
00:07:43,380 --> 00:07:47,460
infrastructure with guidance from the matomo documentation

144
00:07:47,460 --> 00:07:47,940
of course

145
00:07:50,190 --> 00:07:52,170
at the time when we installed the

146
00:07:52,170 --> 00:07:55,770
matomo maria db was still supported by

147
00:07:55,770 --> 00:07:56,370
matomo

148
00:07:57,900 --> 00:08:00,330
and i'm gonna to come back to

149
00:08:00,330 --> 00:08:04,470
that later but we were yeah how

150
00:08:04,470 --> 00:08:08,940
how should we build a bigger application

151
00:08:11,880 --> 00:08:14,760
a monolithic application you could say sir

152
00:08:18,600 --> 00:08:20,430
and i where everything and compiled into

153
00:08:20,430 --> 00:08:22,620
source code and you will put it

154
00:08:22,620 --> 00:08:24,480
on a server and it will just

155
00:08:25,740 --> 00:08:27,690
men sit there and

156
00:08:30,060 --> 00:08:32,100
run the whole application by itself

157
00:08:34,200 --> 00:08:35,940
this which yeah that

158
00:08:36,960 --> 00:08:38,864
the services you have

159
00:08:40,950 --> 00:08:43,260
the things that you see and

160
00:08:44,580 --> 00:08:48,510
the communication layer under the database basically

161
00:08:49,530 --> 00:08:53,130
or microservices so these were the options

162
00:08:53,130 --> 00:08:55,470
we were looking at that time and

163
00:08:55,470 --> 00:08:56,220
of christ

164
00:08:58,710 --> 00:09:01,770
we can't deny that we had requirements

165
00:09:01,770 --> 00:09:02,880
from art

166
00:09:05,370 --> 00:09:08,400
and now i'm imagining that there's a

167
00:09:08,400 --> 00:09:10,710
big crowd out there and you'll probably

168
00:09:10,710 --> 00:09:12,000
see a rocky horror picture show or

169
00:09:12,000 --> 00:09:15,480
something where there are just shadows the

170
00:09:15,570 --> 00:09:18,450
boo or whatever so we add for

171
00:09:18,450 --> 00:09:22,710
example backups are backups need to have

172
00:09:22,710 --> 00:09:24,240
redundancy when it's at

173
00:09:24,360 --> 00:09:27,786
security certified hosting will need to have

174
00:09:27,786 --> 00:09:28,020
a

175
00:09:30,660 --> 00:09:32,220
the status of the application

176
00:09:33,930 --> 00:09:37,020
we need our scalability because everyone doesn't

177
00:09:37,020 --> 00:09:39,960
go in to do their fiscal declaration

178
00:09:40,230 --> 00:09:42,000
once a month this is something you

179
00:09:42,000 --> 00:09:43,710
do once a year probably

180
00:09:45,900 --> 00:09:47,250
need to be able to restore the

181
00:09:47,250 --> 00:09:48,810
systems if it goes down

182
00:09:50,310 --> 00:09:52,560
you need to have a certify their

183
00:09:53,160 --> 00:09:56,460
standards need to be able to roll

184
00:09:56,460 --> 00:09:59,610
back and when a high availability so

185
00:09:59,610 --> 00:10:02,040
that people from all over sweden can

186
00:10:02,040 --> 00:10:05,040
go in and do their declaration space

187
00:10:05,100 --> 00:10:05,460
basically

188
00:10:07,425 --> 00:10:07,875
so

189
00:10:09,105 --> 00:10:09,915
we went to her

190
00:10:10,995 --> 00:10:13,785
hosting partner we've been working with for

191
00:10:14,175 --> 00:10:17,775
many years and and them they are

192
00:10:17,775 --> 00:10:19,875
certified kubernetes now only one said if

193
00:10:19,875 --> 00:10:21,765
i could notice in in and in

194
00:10:21,765 --> 00:10:22,785
the nordics at the time

195
00:10:23,955 --> 00:10:29,115
and so there will of course proposing

196
00:10:29,115 --> 00:10:30,605
that we should run the this uncovered

197
00:10:30,605 --> 00:10:33,645
lettuce because it's the next day and

198
00:10:33,945 --> 00:10:35,745
that's it in there

199
00:10:37,635 --> 00:10:41,085
in tech ops we wanted to go

200
00:10:41,085 --> 00:10:43,515
with the cabinet to set up and

201
00:10:43,515 --> 00:10:44,655
for us it was

202
00:10:46,065 --> 00:10:47,355
although a black box you could

203
00:10:50,675 --> 00:10:52,304
it is not the traditional way you

204
00:10:52,304 --> 00:10:54,825
should think of but sir corrupted kiss

205
00:10:55,005 --> 00:10:55,365
so

206
00:10:56,835 --> 00:10:59,040
we have to wade through to some

207
00:10:59,145 --> 00:11:02,175
barriers to work the works in the

208
00:11:02,175 --> 00:11:02,775
digital is

209
00:11:04,305 --> 00:11:07,245
we took some crash courses there in

210
00:11:07,245 --> 00:11:09,465
cuban at this and the

211
00:11:10,815 --> 00:11:12,525
i'm looking back at it now and

212
00:11:12,525 --> 00:11:15,885
i remember i was thinking like it

213
00:11:15,885 --> 00:11:19,725
was it was a big question mark

214
00:11:19,725 --> 00:11:20,865
in my head at the time but

215
00:11:22,365 --> 00:11:24,825
as more we begin to cuba netizen

216
00:11:24,825 --> 00:11:25,455
there

217
00:11:28,065 --> 00:11:30,735
the everything around kubernetes as it everything

218
00:11:30,975 --> 00:11:32,295
became more clearer so

219
00:11:33,765 --> 00:11:35,895
to you you need to throw everything

220
00:11:35,895 --> 00:11:38,625
you know about servers and hosting out

221
00:11:38,835 --> 00:11:40,665
of the window and you need to

222
00:11:41,205 --> 00:11:44,295
rethink how you think about applications and

223
00:11:44,295 --> 00:11:45,045
services

224
00:11:47,025 --> 00:11:49,425
so this is a picture i took

225
00:11:49,425 --> 00:11:51,255
from the cabinet this website and you

226
00:11:51,255 --> 00:11:54,075
can see that the traditional deployment you

227
00:11:54,075 --> 00:11:54,855
have there

228
00:11:56,565 --> 00:11:58,335
something similar to the three tire

229
00:12:00,555 --> 00:12:01,575
architecture

230
00:12:03,315 --> 00:12:05,445
or you can have the the dss

231
00:12:05,445 --> 00:12:07,245
and we have some you could have

232
00:12:07,245 --> 00:12:10,527
docker or some kind of error or

233
00:12:10,527 --> 00:12:11,505
alter scaling

234
00:12:14,565 --> 00:12:15,165
hosting

235
00:12:16,965 --> 00:12:18,615
can't remember the name you call him

236
00:12:18,855 --> 00:12:21,315
but all he could have like running

237
00:12:21,315 --> 00:12:23,715
container like microservices

238
00:12:28,065 --> 00:12:29,595
so if you if you want to

239
00:12:29,595 --> 00:12:33,375
know what cabinet is isn't death to

240
00:12:33,375 --> 00:12:35,145
the cabinet as where patient there is

241
00:12:35,145 --> 00:12:36,765
a lot of bird accommodation you can

242
00:12:36,765 --> 00:12:37,065
read

243
00:12:39,195 --> 00:12:41,835
but basically the things that we get

244
00:12:41,835 --> 00:12:43,185
out of the box here to meet

245
00:12:43,185 --> 00:12:45,315
the requirement as was sir

246
00:12:47,235 --> 00:12:47,565
that

247
00:12:48,765 --> 00:12:51,495
we could have rollbacks autumn fullbacks

248
00:12:52,725 --> 00:12:54,855
we can have a great

249
00:12:56,565 --> 00:12:57,435
rob is

250
00:12:58,515 --> 00:13:01,995
at least we can do

251
00:13:03,315 --> 00:13:05,565
tinder application without taking down the service

252
00:13:05,595 --> 00:13:06,135
so if

253
00:13:07,695 --> 00:13:10,935
scott the for example let's say hundred

254
00:13:11,085 --> 00:13:15,435
data and analysts and they go in

255
00:13:17,085 --> 00:13:19,995
and they work office time you know

256
00:13:20,055 --> 00:13:22,665
nine to five and we don't want

257
00:13:22,665 --> 00:13:24,345
to take down the application yes if

258
00:13:24,345 --> 00:13:26,205
you want to run out if we

259
00:13:26,205 --> 00:13:29,325
wanted advice on configuration so this needs

260
00:13:29,325 --> 00:13:31,515
to be very careful

261
00:13:34,545 --> 00:13:35,565
during upgrades

262
00:13:39,975 --> 00:13:42,555
yeah so at the time

263
00:13:43,665 --> 00:13:45,375
because we were new to kubernetes as

264
00:13:45,375 --> 00:13:47,955
were the partner elastics they come up

265
00:13:47,955 --> 00:13:49,455
with the architecture

266
00:13:51,525 --> 00:13:53,415
so this is what it looked like

267
00:13:53,415 --> 00:13:55,005
from the start we could say we

268
00:13:55,005 --> 00:13:56,595
have changed some some bits

269
00:13:58,035 --> 00:13:59,235
through the years but

270
00:14:00,735 --> 00:14:04,035
and the basic concept still and and

271
00:14:04,035 --> 00:14:04,515
the

272
00:14:07,755 --> 00:14:11,055
so when i user either if you

273
00:14:11,055 --> 00:14:13,665
go into the dashboard as an analyst

274
00:14:14,565 --> 00:14:15,885
do you get to

275
00:14:17,115 --> 00:14:18,015
one lad

276
00:14:18,735 --> 00:14:19,845
container you could say

277
00:14:22,155 --> 00:14:25,485
or if you have installed your tracking

278
00:14:25,485 --> 00:14:27,855
scripts websites to go

279
00:14:29,145 --> 00:14:32,019
the tracking data will be selling into

280
00:14:32,049 --> 00:14:32,865
the us

281
00:14:34,425 --> 00:14:36,255
and so this is how we can

282
00:14:36,255 --> 00:14:38,775
scale up and down depending on where

283
00:14:38,775 --> 00:14:40,575
the lotus in the application

284
00:14:47,385 --> 00:14:48,195
and of course

285
00:14:51,015 --> 00:14:53,085
when when you dissect a big application

286
00:14:53,085 --> 00:14:53,775
like matomo

287
00:14:55,035 --> 00:14:57,465
but then you're going to run into

288
00:14:57,465 --> 00:14:59,055
some challenges

289
00:15:02,475 --> 00:15:06,345
and i would say the biggest challenge

290
00:15:06,345 --> 00:15:10,005
at the time was sad configuring and

291
00:15:10,095 --> 00:15:12,495
the database too

292
00:15:15,735 --> 00:15:17,685
handled all their load that was coming

293
00:15:17,685 --> 00:15:17,865
in

294
00:15:20,445 --> 00:15:22,425
and we could use many of the

295
00:15:22,425 --> 00:15:24,735
community everything was open source by the

296
00:15:24,735 --> 00:15:26,415
way i forgot to mention that but

297
00:15:26,415 --> 00:15:28,635
we are open source company face a

298
00:15:28,635 --> 00:15:30,495
lot so we like opensource

299
00:15:31,695 --> 00:15:33,645
and we keep it

300
00:15:34,965 --> 00:15:36,675
we like to keep our infrastructure as

301
00:15:36,675 --> 00:15:38,565
code and and the best way of

302
00:15:38,565 --> 00:15:40,605
doing that at a time with the

303
00:15:41,595 --> 00:15:44,085
help from our partner elastics when they

304
00:15:44,085 --> 00:15:46,215
come up with hell so helm is

305
00:15:46,219 --> 00:15:48,945
a way to keep your

306
00:15:50,295 --> 00:15:52,635
charts as they call them the templates

307
00:15:52,635 --> 00:15:55,485
for your infrastructure and so this is

308
00:15:55,485 --> 00:15:58,005
how we can easily tweak bits and

309
00:15:58,005 --> 00:16:01,605
pieces of the application and without having

310
00:16:01,605 --> 00:16:02,295
to

311
00:16:03,705 --> 00:16:05,685
the access to a server and in

312
00:16:05,685 --> 00:16:08,985
manually and change something so every change

313
00:16:08,985 --> 00:16:11,715
needs to be documented so we can

314
00:16:11,715 --> 00:16:13,785
do this in our version controlling system

315
00:16:13,875 --> 00:16:14,445
with git

316
00:16:20,385 --> 00:16:22,695
yeah so this is hell

317
00:16:25,845 --> 00:16:29,685
and of course we have a

318
00:16:30,915 --> 00:16:34,515
require mass as like cats and ship

319
00:16:34,515 --> 00:16:37,725
out code deploy code that have vulnerabilities

320
00:16:37,845 --> 00:16:40,155
in the software to figure out some

321
00:16:40,155 --> 00:16:41,745
kind of id see i see the

322
00:16:41,745 --> 00:16:44,925
workflow where we can ultimate stuff because

323
00:16:44,925 --> 00:16:47,085
you will you you can really track

324
00:16:47,085 --> 00:16:49,605
all the vulnerabilities that come up all

325
00:16:49,605 --> 00:16:51,045
the time so

326
00:16:51,045 --> 00:16:53,475
the way automates as much as possible

327
00:16:55,005 --> 00:16:56,325
in the cut

328
00:16:58,275 --> 00:16:59,835
we have a drone

329
00:17:01,665 --> 00:17:04,935
a drone is a cia service

330
00:17:06,585 --> 00:17:08,234
where you can plug in a different

331
00:17:08,234 --> 00:17:10,785
kind of applications to a third party

332
00:17:10,785 --> 00:17:13,065
system that will check your code your

333
00:17:13,065 --> 00:17:15,585
base code so if you're peckish all

334
00:17:15,585 --> 00:17:18,375
your motto more into a container in

335
00:17:18,375 --> 00:17:19,305
docker let's say

336
00:17:21,194 --> 00:17:24,974
and all the underlying system applications

337
00:17:26,954 --> 00:17:28,484
that are in this docker image

338
00:17:29,835 --> 00:17:30,525
drone

339
00:17:32,175 --> 00:17:34,695
will stop service and then we will

340
00:17:34,695 --> 00:17:36,525
use something

341
00:17:38,475 --> 00:17:39,585
that is called claire

342
00:17:40,584 --> 00:17:41,595
just the next line

343
00:17:43,065 --> 00:17:46,065
here and also it's an open source

344
00:17:46,065 --> 00:17:47,925
service that will check for vulnerabilities

345
00:17:49,395 --> 00:17:55,155
so everything is running containers so before

346
00:17:55,155 --> 00:17:56,925
we deploy something out to

347
00:17:58,086 --> 00:18:01,733
production site this will run up a

348
00:18:01,755 --> 00:18:04,095
similar environment

349
00:18:06,525 --> 00:18:08,535
where it will check for vulnerabilities and

350
00:18:08,535 --> 00:18:10,005
when we get a green light then

351
00:18:10,005 --> 00:18:11,595
we can do to deploy to the

352
00:18:11,595 --> 00:18:16,125
production and of course being an open

353
00:18:16,125 --> 00:18:18,555
source company and how much the community

354
00:18:18,585 --> 00:18:21,345
means for us we have also open

355
00:18:21,345 --> 00:18:24,945
sourced our hell base set up

356
00:18:25,455 --> 00:18:28,515
so anyone can take our work and

357
00:18:29,265 --> 00:18:31,935
replicate what we have done and if

358
00:18:31,935 --> 00:18:33,585
you go to our github you can

359
00:18:33,585 --> 00:18:35,205
find the matomo cabinet this

360
00:18:36,495 --> 00:18:38,355
they set up the we are using

361
00:18:41,865 --> 00:18:45,765
of course there there's other dependencies so

362
00:18:45,765 --> 00:18:47,235
we use this as a base and

363
00:18:47,235 --> 00:18:49,515
then we have there a different

364
00:18:51,652 --> 00:18:55,305
a project where we the package

365
00:18:56,565 --> 00:18:57,435
and replant well

366
00:19:03,285 --> 00:19:04,095
the worst

367
00:19:05,558 --> 00:19:08,415
dissecting an application like matomo

368
00:19:10,455 --> 00:19:14,175
you want to have the control you

369
00:19:14,175 --> 00:19:15,855
want to automate as much as possible

370
00:19:15,855 --> 00:19:17,955
so when we are dependent on the

371
00:19:17,985 --> 00:19:20,745
cla tools which are

372
00:19:22,275 --> 00:19:24,915
at the time we couldn't use we

373
00:19:24,915 --> 00:19:27,675
can add any new users or so

374
00:19:28,155 --> 00:19:31,005
every deploy basically we will destroy everything

375
00:19:31,935 --> 00:19:35,085
and we build a home more and

376
00:19:35,115 --> 00:19:37,335
if you know metal man how to

377
00:19:37,335 --> 00:19:37,935
install it

378
00:19:39,255 --> 00:19:41,595
basically it's a clean installer every everytime

379
00:19:42,105 --> 00:19:45,543
so to be able to keep an

380
00:19:46,125 --> 00:19:49,665
our database without wiping it and we

381
00:19:49,665 --> 00:19:50,655
had to come up with

382
00:19:51,795 --> 00:19:54,315
different seo tools to automate some of

383
00:19:54,315 --> 00:19:56,685
the stuff so and were contributed this

384
00:19:56,685 --> 00:19:59,475
back to their community and we have

385
00:19:59,505 --> 00:20:03,015
summer plug plugins in the matomo marketplace

386
00:20:03,525 --> 00:20:04,934
one of them will use is the

387
00:20:04,934 --> 00:20:05,805
use of console

388
00:20:07,605 --> 00:20:10,305
but also more most importantly i would

389
00:20:10,305 --> 00:20:11,715
say the extra tools

390
00:20:13,705 --> 00:20:16,945
where we can ultimate a lot of

391
00:20:16,945 --> 00:20:21,295
things like year db backups or installation

392
00:20:21,625 --> 00:20:22,285
of matomo

393
00:20:24,415 --> 00:20:24,865
and

394
00:20:26,155 --> 00:20:28,285
i took this screenshot yesterday and it's

395
00:20:28,285 --> 00:20:32,185
been downloaded over four thousand nine hundred

396
00:20:32,635 --> 00:20:35,395
times i think that's fantastic

397
00:20:43,885 --> 00:20:46,765
yeah and this is there more

398
00:20:49,615 --> 00:20:53,305
abstract or review of our application this

399
00:20:53,305 --> 00:20:54,745
is more about how pumping

400
00:20:56,095 --> 00:20:56,365
that

401
00:20:59,635 --> 00:21:03,025
so if inconveniences were were you haven't

402
00:21:03,445 --> 00:21:05,725
called a pod and a pulled can

403
00:21:05,725 --> 00:21:07,345
run multiple containers

404
00:21:10,735 --> 00:21:14,125
and you can have a different pause

405
00:21:14,635 --> 00:21:16,015
the asked

406
00:21:18,325 --> 00:21:19,885
so for example if you want to

407
00:21:19,915 --> 00:21:22,675
run the core archive job as sn

408
00:21:23,935 --> 00:21:26,425
the presentation before me and

409
00:21:27,595 --> 00:21:29,785
he mentioned the core archive in there

410
00:21:29,845 --> 00:21:31,225
if you go into the server and

411
00:21:31,225 --> 00:21:32,483
you used to say like to he

412
00:21:32,483 --> 00:21:35,436
can run the archiver command and

413
00:21:36,715 --> 00:21:37,615
in a cronjob

414
00:21:39,325 --> 00:21:41,845
in kubernetes this world a this is

415
00:21:41,845 --> 00:21:42,835
called a

416
00:21:46,525 --> 00:21:48,355
they're a job

417
00:21:49,405 --> 00:21:50,515
just a job there

418
00:21:51,685 --> 00:21:52,375
so

419
00:21:54,535 --> 00:21:59,065
and because of the high load or

420
00:22:00,115 --> 00:22:02,455
the millions of page views every month

421
00:22:02,575 --> 00:22:06,355
and not a single corker care jobs

422
00:22:06,985 --> 00:22:09,685
wasn't enough for us so we tried

423
00:22:09,685 --> 00:22:11,005
to come up with different ways of

424
00:22:11,005 --> 00:22:13,615
running this which has led into

425
00:22:13,645 --> 00:22:16,105
other problems too and we have learned

426
00:22:16,195 --> 00:22:19,285
a lot during the years so

427
00:22:20,365 --> 00:22:22,225
and we have had discussions with the

428
00:22:22,225 --> 00:22:24,205
moto moto him to how to solve

429
00:22:24,205 --> 00:22:26,755
this properly in the correct way and

430
00:22:27,205 --> 00:22:29,365
yet i bet to have run into

431
00:22:29,365 --> 00:22:32,095
similar issues like we have and the

432
00:22:32,155 --> 00:22:33,775
but they have probably solved currently

433
00:22:38,425 --> 00:22:41,035
yeah of course monitoring is important to

434
00:22:41,035 --> 00:22:43,075
know when you need to scale up

435
00:22:43,075 --> 00:22:45,205
or scale scale down your application so

436
00:22:45,205 --> 00:22:46,645
of course with you sir

437
00:22:47,755 --> 00:22:50,215
with communities is a very standard to

438
00:22:50,215 --> 00:22:54,265
use the prometheus that will check the

439
00:22:56,125 --> 00:22:58,855
the paris and we use grafana to

440
00:22:58,855 --> 00:23:00,685
visualize are the metrics

441
00:23:02,335 --> 00:23:04,255
which is another story

442
00:23:06,055 --> 00:23:07,915
they're like kibana if you use to

443
00:23:07,915 --> 00:23:09,355
keep an hour elastic stack

444
00:23:10,945 --> 00:23:15,145
i said yeah

445
00:23:16,285 --> 00:23:18,595
jumping into cabinet this was a lot

446
00:23:18,595 --> 00:23:20,155
of fun but it's a

447
00:23:22,075 --> 00:23:26,005
it's it's it's alot to begin to

448
00:23:26,575 --> 00:23:27,805
because you need to know

449
00:23:29,185 --> 00:23:32,755
about a lot of stuff so

450
00:23:33,895 --> 00:23:34,315
fun

451
00:23:35,815 --> 00:23:36,145
yeah

452
00:23:37,405 --> 00:23:39,475
so just to give you a hint

453
00:23:39,475 --> 00:23:42,115
of how much how many cases we're

454
00:23:42,115 --> 00:23:44,545
dealing with here so i tweaked the

455
00:23:44,545 --> 00:23:46,525
numbers are lopez so i am not

456
00:23:46,555 --> 00:23:47,755
i don't want to give away their

457
00:23:47,815 --> 00:23:51,145
real numbers but this is very like

458
00:23:51,865 --> 00:23:52,555
the real deal

459
00:23:53,785 --> 00:23:56,935
we have also other clients with even

460
00:23:56,935 --> 00:23:59,575
more data than this so we know

461
00:23:59,575 --> 00:24:01,255
what we're doing we have done this

462
00:24:01,255 --> 00:24:02,515
for many years now

463
00:24:04,945 --> 00:24:06,835
and if you look here the fiscal

464
00:24:06,835 --> 00:24:09,685
declaration for the swedish texts

465
00:24:11,095 --> 00:24:14,545
there's it's opens up in mars and

466
00:24:14,545 --> 00:24:17,005
you can see there's a peak because

467
00:24:17,005 --> 00:24:18,775
everyone wants to go inland to see

468
00:24:18,775 --> 00:24:21,055
their tax returns if they're getting anything

469
00:24:21,055 --> 00:24:23,785
i still have to pay more taxes

470
00:24:23,875 --> 00:24:26,695
so this is something we can prepare

471
00:24:26,695 --> 00:24:28,045
for and we can scale up our

472
00:24:28,053 --> 00:24:28,765
services

473
00:24:31,015 --> 00:24:33,775
we can also under the

474
00:24:36,355 --> 00:24:37,675
at under the hood we can also

475
00:24:37,705 --> 00:24:38,455
add more

476
00:24:41,245 --> 00:24:43,675
servers are them as hardware if you

477
00:24:43,675 --> 00:24:46,765
want so that's possible too so when

478
00:24:47,995 --> 00:24:51,265
but during migration and that's something i

479
00:24:51,265 --> 00:24:53,065
would not recommend for him once said

480
00:24:53,305 --> 00:24:55,045
if you want been a busy metonymy

481
00:24:55,045 --> 00:24:58,855
skin and so you have enough hardware

482
00:24:58,855 --> 00:25:00,895
in place because doing a my migration

483
00:25:00,895 --> 00:25:02,995
is a tough project

484
00:25:07,375 --> 00:25:10,045
and so here's a little peek from

485
00:25:10,135 --> 00:25:12,385
one of our town where we override

486
00:25:12,475 --> 00:25:14,785
the values in their home shirts so

487
00:25:14,785 --> 00:25:18,295
this is something where we can prepare

488
00:25:18,385 --> 00:25:20,695
like is the fiscal declaration is coming

489
00:25:20,695 --> 00:25:22,075
up in his opening we know that

490
00:25:22,135 --> 00:25:24,175
we can we can add more replica

491
00:25:24,175 --> 00:25:24,445
us

492
00:25:26,455 --> 00:25:27,955
to the dashboard or to the tracker

493
00:25:27,955 --> 00:25:28,495
pods

494
00:25:31,885 --> 00:25:35,155
and if behind the red lion is

495
00:25:35,155 --> 00:25:36,265
yours of the constant one of the

496
00:25:36,265 --> 00:25:38,815
console commands this is also one of

497
00:25:38,815 --> 00:25:42,265
the challenges we have because we couldn't

498
00:25:42,415 --> 00:25:43,885
at the time we couldn't use a

499
00:25:44,335 --> 00:25:48,775
rewrite many an nfl folder and so

500
00:25:48,835 --> 00:25:49,847
we had to come up with

501
00:25:49,855 --> 00:25:52,495
the idea how to regenerate

502
00:25:54,025 --> 00:25:55,765
tag manager containers

503
00:25:57,055 --> 00:25:57,429
so

504
00:25:59,635 --> 00:26:01,945
a way you can do that in

505
00:26:01,945 --> 00:26:04,615
the cabinet has set up mr run

506
00:26:04,615 --> 00:26:06,625
the command so this is where the

507
00:26:06,625 --> 00:26:08,635
console commands are so important for us

508
00:26:11,305 --> 00:26:14,035
yeah we can also add more cpu

509
00:26:14,095 --> 00:26:18,235
more memory to each pod or group

510
00:26:18,235 --> 00:26:20,335
of pods yeah i wouldn't recommend to

511
00:26:20,335 --> 00:26:22,405
do it individually per pod but you

512
00:26:22,405 --> 00:26:23,485
can learn yoga

513
00:26:25,465 --> 00:26:28,675
the basic template and then he can

514
00:26:28,675 --> 00:26:31,675
tweak that for a different services

515
00:26:33,745 --> 00:26:37,240
we have one that is sad tracking

516
00:26:37,240 --> 00:26:37,675
year

517
00:26:38,861 --> 00:26:40,375
the tracking data is sent into i

518
00:26:40,375 --> 00:26:43,255
read the middle layer where it's stores

519
00:26:43,315 --> 00:26:45,655
because the if the database is too

520
00:26:45,655 --> 00:26:48,327
busy we need to keep the data

521
00:26:48,327 --> 00:26:50,965
so it doesn't disappear or take down

522
00:26:50,965 --> 00:26:52,525
the database so this is how we

523
00:26:52,525 --> 00:26:52,825
can

524
00:26:55,525 --> 00:26:57,595
the cure tracking service

525
00:27:00,715 --> 00:27:03,595
and is this a peek at the

526
00:27:03,595 --> 00:27:07,135
name we would only use satellite tools

527
00:27:07,135 --> 00:27:10,105
but now the bigger we get and

528
00:27:10,105 --> 00:27:11,485
the bigger the team gets we need

529
00:27:11,485 --> 00:27:13,435
to ultimate level stuff this is a

530
00:27:14,335 --> 00:27:17,275
dream or i see i see the

531
00:27:17,275 --> 00:27:18,115
workflows

532
00:27:19,105 --> 00:27:21,620
using the tools there and there's called

533
00:27:21,788 --> 00:27:25,645
std which i think is really awesome

534
00:27:25,735 --> 00:27:27,295
also open source of course

535
00:27:28,585 --> 00:27:30,265
where we can now go in and

536
00:27:30,775 --> 00:27:32,815
see all the parts in a nice

537
00:27:32,815 --> 00:27:36,985
graphic view arrow and can check logs

538
00:27:36,985 --> 00:27:39,295
or we can see the health status

539
00:27:39,295 --> 00:27:42,895
and greenheart means that everything is operational

540
00:27:42,895 --> 00:27:46,705
and running late you see the magnitude

541
00:27:46,705 --> 00:27:48,595
of the pods are the microservices

542
00:27:48,655 --> 00:27:51,115
there is running in the background and

543
00:27:54,085 --> 00:27:57,085
yeah and it is an attitude that

544
00:27:57,085 --> 00:27:59,185
i like that i've been using a

545
00:27:59,185 --> 00:28:01,045
lot through the years he said jay

546
00:28:01,058 --> 00:28:07,795
nice and it's a nice cla visualization

547
00:28:07,795 --> 00:28:10,525
tool to see the health and status

548
00:28:10,525 --> 00:28:12,025
of your cluster

549
00:28:15,025 --> 00:28:17,005
so i can highly recommend to use

550
00:28:17,005 --> 00:28:20,095
canines if you're more into sitting in

551
00:28:20,095 --> 00:28:22,105
a terminal than having a graphical overview

552
00:28:24,925 --> 00:28:27,835
yeah so the challenges we had through

553
00:28:27,835 --> 00:28:31,075
the years was the db configuration and

554
00:28:31,075 --> 00:28:35,845
then jumping into trying to debug a

555
00:28:35,845 --> 00:28:38,545
black box from the beginning to find

556
00:28:38,665 --> 00:28:42,295
as throttles or bottlenecks in our application

557
00:28:46,195 --> 00:28:47,875
yeah it was sir

558
00:28:49,075 --> 00:28:51,475
yeah some of the ways we do

559
00:28:51,475 --> 00:28:53,635
now we have the added more

560
00:28:55,495 --> 00:28:57,805
just shared all of those to the

561
00:28:57,805 --> 00:29:00,835
community yet because they are not

562
00:29:02,515 --> 00:29:04,975
ready for taste i would say but

563
00:29:06,025 --> 00:29:08,185
we get enough of data to to

564
00:29:08,185 --> 00:29:11,065
find if if we went around the

565
00:29:11,935 --> 00:29:13,855
moscow journal to check the health of

566
00:29:13,855 --> 00:29:16,135
the database we will have automated a

567
00:29:16,135 --> 00:29:17,635
lot of stops to do that mothers

568
00:29:18,505 --> 00:29:21,742
and from the beginning maria db being

569
00:29:21,775 --> 00:29:22,015
opened

570
00:29:22,015 --> 00:29:23,908
source company and the fact that the

571
00:29:23,935 --> 00:29:27,205
mice girl wasn't no longer open sorcery

572
00:29:27,205 --> 00:29:29,635
was open-source but kind of

573
00:29:30,715 --> 00:29:33,025
because they were acquired by oracle

574
00:29:35,335 --> 00:29:38,815
we chose maria db and when matomo

575
00:29:38,995 --> 00:29:40,255
added more features

576
00:29:42,835 --> 00:29:45,085
two thousand and nineteen i think to

577
00:29:45,655 --> 00:29:48,745
run big madame installations they wouldn't support

578
00:29:48,745 --> 00:29:51,925
maria db and that was a very

579
00:29:51,925 --> 00:29:55,945
frustrating sometimes i did a patch so

580
00:29:55,975 --> 00:29:59,845
i can do stuff now today though

581
00:30:00,625 --> 00:30:01,585
we couldn't do a maria

582
00:30:01,645 --> 00:30:02,875
he'd be back back then

583
00:30:04,195 --> 00:30:06,325
and then of course the core archives

584
00:30:06,505 --> 00:30:08,755
are having a lot of data and

585
00:30:08,755 --> 00:30:09,175
then

586
00:30:10,375 --> 00:30:12,265
one of my colleagues tomas is going

587
00:30:12,265 --> 00:30:14,815
to talk more about some of the

588
00:30:15,355 --> 00:30:18,376
challenges we had had or how how

589
00:30:18,409 --> 00:30:19,884
you can break metal i think his

590
00:30:19,884 --> 00:30:22,644
session name is the title of his

591
00:30:22,644 --> 00:30:23,125
session

592
00:30:24,625 --> 00:30:27,085
i would recommend to join his sessions

593
00:30:27,085 --> 00:30:27,805
because sir

594
00:30:28,735 --> 00:30:30,745
yeah he he will talk more in

595
00:30:30,745 --> 00:30:34,585
detail how how to break puerto rico's

596
00:30:35,455 --> 00:30:37,765
and there we problems from the beginning

597
00:30:37,765 --> 00:30:41,575
with forms that they would automatically include

598
00:30:41,575 --> 00:30:42,325
in every

599
00:30:43,645 --> 00:30:45,776
now recite that you put up in

600
00:30:45,776 --> 00:30:46,495
there matomo

601
00:30:47,665 --> 00:30:49,525
and that gave us a little bit

602
00:30:49,525 --> 00:30:50,485
of headache mo

603
00:30:51,779 --> 00:30:53,335
we have dealt with that

604
00:30:54,955 --> 00:30:55,255
fixed

605
00:30:56,515 --> 00:30:58,585
and the segments go hand in hand

606
00:30:58,585 --> 00:31:00,445
with core archives i would say because

607
00:31:05,335 --> 00:31:07,686
the motto more features in the u

608
00:31:07,686 --> 00:31:08,125
why

609
00:31:09,295 --> 00:31:11,725
and having a lot of analysts analysts

610
00:31:11,785 --> 00:31:14,995
working there and morne putting up their

611
00:31:14,995 --> 00:31:17,365
own segments with squares that are

612
00:31:19,375 --> 00:31:22,195
they're getting a lot of data from

613
00:31:22,195 --> 00:31:24,715
the database could potentially take down your

614
00:31:24,715 --> 00:31:25,345
application

615
00:31:27,235 --> 00:31:29,325
and of course the transitions

616
00:31:31,435 --> 00:31:33,235
because it's squaring their lives

617
00:31:35,575 --> 00:31:37,735
and add indexes to the database and

618
00:31:39,475 --> 00:31:41,905
yeah also i don't think you think

619
00:31:41,905 --> 00:31:43,105
we still have some problem with the

620
00:31:43,105 --> 00:31:43,945
transitions

621
00:31:45,745 --> 00:31:48,985
heatmaps session recordings i would say it

622
00:31:48,985 --> 00:31:52,405
works very well very well sometimes but

623
00:31:52,405 --> 00:31:54,715
sometimes it's it could spam the logs

624
00:31:54,715 --> 00:31:56,727
or we have one of our clients

625
00:31:56,727 --> 00:31:57,625
is using a

626
00:31:58,735 --> 00:32:01,615
a network monitoring series that will

627
00:32:03,055 --> 00:32:04,855
filled with the traffic that it will

628
00:32:04,855 --> 00:32:08,095
detect as a spammer or a denial

629
00:32:08,095 --> 00:32:10,705
of service attacks or whatever so this

630
00:32:10,705 --> 00:32:15,113
is something we didn't foresee and when

631
00:32:15,115 --> 00:32:17,185
one of the class activated heatmaps and

632
00:32:17,185 --> 00:32:18,145
session recordings

633
00:32:19,495 --> 00:32:20,605
so that was interesting

634
00:32:24,775 --> 00:32:27,595
so a quick recap because i think

635
00:32:27,595 --> 00:32:28,885
my time is running out here

636
00:32:31,465 --> 00:32:34,555
yeah so we use docker toma has

637
00:32:34,645 --> 00:32:36,865
a a docker template if you go

638
00:32:36,865 --> 00:32:39,955
into the official matomo github webpage you

639
00:32:39,955 --> 00:32:41,575
can search for their docker

640
00:32:42,925 --> 00:32:44,905
we have were of course hosting our

641
00:32:44,905 --> 00:32:47,245
own docker template but i would say

642
00:32:47,245 --> 00:32:48,655
that's a good start if you want

643
00:32:48,655 --> 00:32:50,575
to run but only in a docker

644
00:32:50,845 --> 00:32:52,375
set up in a cluster environment

645
00:32:56,545 --> 00:33:00,325
and the yeah cuban at this i

646
00:33:00,325 --> 00:33:02,995
have talked about a lot any questions

647
00:33:03,145 --> 00:33:07,255
feel free letter that says i have

648
00:33:07,255 --> 00:33:08,635
ten minutes have to

649
00:33:09,835 --> 00:33:12,625
yeah drawn to automate stuff when we

650
00:33:12,715 --> 00:33:15,205
push our code into guitar or gitlab

651
00:33:15,475 --> 00:33:17,425
or github is actually running their own

652
00:33:17,425 --> 00:33:20,005
seo service but drawing is something you

653
00:33:20,005 --> 00:33:23,155
can integrate with your github repos and

654
00:33:23,155 --> 00:33:26,695
then clear to check for vulnerabilities in

655
00:33:26,695 --> 00:33:29,845
your in your images

656
00:33:30,355 --> 00:33:31,405
in your docker images

657
00:33:33,865 --> 00:33:35,785
github gitlab i mention

658
00:33:37,495 --> 00:33:39,355
there's probably something i have forgotten

659
00:33:41,515 --> 00:33:42,475
but i think you get the big

660
00:33:42,475 --> 00:33:43,075
picture

661
00:33:44,335 --> 00:33:45,925
and he was an interesting yearning i

662
00:33:45,925 --> 00:33:46,405
would say

663
00:33:48,145 --> 00:33:50,545
and i will leave room for questions

664
00:33:50,575 --> 00:33:52,285
and i will see if i can

665
00:33:52,375 --> 00:33:53,995
open up the chats here

666
00:34:18,025 --> 00:34:21,085
mckay is fifty

667
00:34:23,695 --> 00:34:26,005
hey jorge i've shared some of the

668
00:34:26,005 --> 00:34:27,955
questions here in the chat

669
00:34:29,185 --> 00:34:30,475
yeah yeah

670
00:34:39,085 --> 00:34:41,962
from valentine inertia thanks for contributing to

671
00:34:41,965 --> 00:34:44,275
the source with this ripple and you're

672
00:34:44,275 --> 00:34:44,935
welcome

673
00:34:46,284 --> 00:34:48,385
question have you ever used maternal fake

674
00:34:48,385 --> 00:34:50,034
visitor plugin in order to have enough

675
00:34:50,034 --> 00:34:51,685
data within your maternal death in order

676
00:34:51,685 --> 00:34:53,725
to make some crash tests do you

677
00:34:54,085 --> 00:34:56,815
think he makes some crash test with

678
00:34:56,905 --> 00:34:59,575
empty automall database the idea would be

679
00:34:59,575 --> 00:35:03,355
to educate ourselves about system administration

680
00:35:04,945 --> 00:35:08,305
ah yes i have used a fake

681
00:35:08,305 --> 00:35:13,195
research plugin and yes i wouldn't say

682
00:35:13,285 --> 00:35:16,135
that it's the best option to test

683
00:35:16,285 --> 00:35:21,325
a big dataset because it takes a

684
00:35:21,355 --> 00:35:22,795
very long time to

685
00:35:24,115 --> 00:35:27,295
i get enough of the fake visitors

686
00:35:28,045 --> 00:35:31,675
so some of the annotation tests are

687
00:35:31,675 --> 00:35:34,855
the love the band we have

688
00:35:36,145 --> 00:35:39,325
actually had taken a big

689
00:35:40,465 --> 00:35:41,095
yeah

690
00:35:42,235 --> 00:35:46,555
dump of food of course with them

691
00:35:48,655 --> 00:35:51,409
from production to because we want to

692
00:35:51,409 --> 00:35:54,355
have a slyke production as possible

693
00:35:56,005 --> 00:35:58,675
but i would say there's still we

694
00:35:58,675 --> 00:35:59,455
haven't figured out the

695
00:36:00,625 --> 00:36:04,435
the perfect way to do this thing

696
00:36:09,145 --> 00:36:11,605
if i could give an advice

697
00:36:13,585 --> 00:36:15,625
yeah there's some

698
00:36:18,055 --> 00:36:20,875
there's a application i can remember the

699
00:36:20,875 --> 00:36:24,745
name right now this used a lot

700
00:36:24,745 --> 00:36:27,775
too at low test the varnish for

701
00:36:27,775 --> 00:36:28,375
example

702
00:36:31,345 --> 00:36:32,935
i can if it if your ping

703
00:36:32,935 --> 00:36:34,585
me i talk to you

704
00:36:36,535 --> 00:36:38,455
in linkedin or twitter i will share

705
00:36:38,455 --> 00:36:39,355
to my

706
00:36:41,005 --> 00:36:41,875
links later

707
00:36:43,975 --> 00:36:45,955
okay question from her better with millions

708
00:36:46,015 --> 00:36:47,635
of page views how big is your

709
00:36:47,635 --> 00:36:50,065
database have you ever met issues regarding

710
00:36:50,065 --> 00:36:51,685
the size of your db and the

711
00:36:51,685 --> 00:36:56,568
archived process so as to be the

712
00:36:56,575 --> 00:36:57,625
case i would say

713
00:37:01,165 --> 00:37:02,605
eyes noise

714
00:37:03,955 --> 00:37:06,415
i would say around it steady around

715
00:37:06,505 --> 00:37:08,454
eight hundred gigabytes or one terabyte i

716
00:37:08,454 --> 00:37:10,045
would say it goes up and down

717
00:37:10,045 --> 00:37:12,175
when we purchased sunday data sometimes they're

718
00:37:12,295 --> 00:37:14,832
after and our carvings are down some

719
00:37:14,832 --> 00:37:16,675
man some of our

720
00:37:18,498 --> 00:37:22,703
clients have different requirements for example that

721
00:37:22,703 --> 00:37:23,905
they want they want to

722
00:37:25,345 --> 00:37:27,895
do some are acquiring to the database

723
00:37:28,165 --> 00:37:30,745
one year after of course that database

724
00:37:30,745 --> 00:37:31,285
will grow

725
00:37:32,695 --> 00:37:36,355
very big but the class only store

726
00:37:36,355 --> 00:37:38,095
for one hundred and eighty days so

727
00:37:38,815 --> 00:37:42,085
we can start to if delete old

728
00:37:42,145 --> 00:37:44,185
logs after one hundred days and that

729
00:37:44,185 --> 00:37:46,645
saves more if that's better for the

730
00:37:46,645 --> 00:37:47,845
application of levity

731
00:37:50,755 --> 00:37:53,185
and you can also integrate matomo to

732
00:37:53,905 --> 00:37:54,865
a data lake

733
00:37:56,575 --> 00:37:59,035
a duke hadoop cluster or

734
00:38:01,345 --> 00:38:03,122
kafka or something if you want to

735
00:38:03,205 --> 00:38:05,725
be serious or superset it's something we

736
00:38:05,725 --> 00:38:07,225
have been looking into

737
00:38:11,215 --> 00:38:13,915
if you want to do more pivoting

738
00:38:13,915 --> 00:38:16,945
of the dataset some asking big queries

739
00:38:17,005 --> 00:38:21,655
to the database and so yeah and

740
00:38:21,655 --> 00:38:23,575
the archiving process yeah we have had

741
00:38:23,575 --> 00:38:25,075
a lot of issues i think we

742
00:38:25,075 --> 00:38:26,845
have if you want to follow our

743
00:38:27,325 --> 00:38:28,375
issues we have had

744
00:38:28,465 --> 00:38:29,785
through the years you can go into

745
00:38:29,785 --> 00:38:31,225
the moto mais q and he can

746
00:38:31,225 --> 00:38:34,105
search for a correr kaveri we have

747
00:38:34,105 --> 00:38:36,415
had but

748
00:38:38,515 --> 00:38:40,645
i think we have reached a level

749
00:38:41,095 --> 00:38:43,525
of acceptance now and one of the

750
00:38:43,525 --> 00:38:45,115
challenges i think thomas is going to

751
00:38:45,115 --> 00:38:49,195
talk more about and it's about educating

752
00:38:49,195 --> 00:38:53,425
your analysts how to build segments

753
00:38:58,154 --> 00:39:01,495
what else the forms you have goals

754
00:39:01,495 --> 00:39:06,295
and all those different plugins or applications

755
00:39:06,445 --> 00:39:09,415
or the niece or the aura some

756
00:39:09,415 --> 00:39:12,385
kind of archiving and because if you

757
00:39:12,385 --> 00:39:14,725
do too big query on a big

758
00:39:14,815 --> 00:39:15,025
day

759
00:39:15,025 --> 00:39:19,165
dataset with all type of monolithic good

760
00:39:19,194 --> 00:39:20,275
database system

761
00:39:21,745 --> 00:39:23,875
i already know what is it called

762
00:39:25,045 --> 00:39:27,925
a tabular database like my fellow maria

763
00:39:27,925 --> 00:39:31,525
db and you you would eventually hit

764
00:39:31,525 --> 00:39:35,305
some problems or bottlenecks with us you

765
00:39:35,305 --> 00:39:38,875
can always optimize the conflagration but i

766
00:39:38,875 --> 00:39:40,435
think if you want to do big

767
00:39:40,735 --> 00:39:41,665
queries

768
00:39:41,965 --> 00:39:42,145
on

769
00:39:43,915 --> 00:39:46,045
database you will need to integrate with

770
00:39:46,045 --> 00:39:48,715
some other systems like superset that i

771
00:39:48,715 --> 00:39:49,435
just mentioned

772
00:39:50,545 --> 00:39:50,785
one

773
00:39:53,635 --> 00:39:54,355
what else

774
00:39:56,935 --> 00:39:58,836
perhaps i missed a number of events

775
00:39:58,855 --> 00:40:00,595
and scattered i get the figures showed

776
00:40:00,595 --> 00:40:03,115
were purged page views right yes those

777
00:40:03,115 --> 00:40:03,475
correct

778
00:40:05,485 --> 00:40:05,665
ah

779
00:40:06,085 --> 00:40:08,635
i dunno in from the top of

780
00:40:08,635 --> 00:40:10,315
my head i dunno their lands but

781
00:40:10,405 --> 00:40:12,085
a sense because sir

782
00:40:13,615 --> 00:40:17,785
and they are using the tag manager

783
00:40:17,785 --> 00:40:20,365
as much as others some have utter

784
00:40:20,365 --> 00:40:22,555
of plants that uses a lot of

785
00:40:22,555 --> 00:40:24,085
single page applications

786
00:40:25,715 --> 00:40:29,045
then many it doesn't differentiate that much

787
00:40:29,375 --> 00:40:31,355
of course there's a there's more events

788
00:40:31,355 --> 00:40:34,327
of course i could probably look the

789
00:40:34,745 --> 00:40:35,945
him in a minute

790
00:40:42,185 --> 00:40:43,565
i can jump to the next question

791
00:40:46,805 --> 00:40:48,845
is maria db not supporting maternal or

792
00:40:48,845 --> 00:40:51,335
isn't recommended for large installations at metal

793
00:40:52,835 --> 00:40:55,805
yeah so what didn't

794
00:40:57,395 --> 00:41:00,905
the supporting of my maria db am

795
00:41:02,717 --> 00:41:05,015
was it two years ago and

796
00:41:06,575 --> 00:41:08,405
i would say you can still install

797
00:41:08,435 --> 00:41:11,225
with maria db of corridors for small

798
00:41:11,225 --> 00:41:14,195
sites but for large installations if you

799
00:41:14,195 --> 00:41:16,715
want to use some configuration such as

800
00:41:16,715 --> 00:41:17,225
sir

801
00:41:21,494 --> 00:41:23,525
there are some nice girl specific

802
00:41:26,045 --> 00:41:28,565
configurations though not always using

803
00:41:30,845 --> 00:41:32,285
and i think my time is up

804
00:41:32,345 --> 00:41:34,595
there and but whereas you can reach

805
00:41:34,595 --> 00:41:38,735
out and somehow i will share my

806
00:41:38,735 --> 00:41:41,315
slice if you want to get in

807
00:41:41,315 --> 00:41:43,085
touch with me or ask more questions

808
00:41:43,145 --> 00:41:44,825
and i will be more than happy

809
00:41:44,825 --> 00:41:46,145
to share them with you

810
00:41:49,415 --> 00:41:50,705
so thank you for your time i

811
00:41:50,705 --> 00:41:52,085
thank you for listening and thank you

812
00:41:52,085 --> 00:41:53,915
for the madame kept him awesome