1 00:00:00,000 --> 00:00:11,120 Hello everyone and welcome to this second day of the Matomo Camp. 2 00:00:11,120 --> 00:00:16,400 So for the moment everything is going well, I hope that will be the case throughout this 3 00:00:16,400 --> 00:00:23,320 second day. We are lucky and honored today to receive Valentin Bares who works 4 00:00:23,320 --> 00:00:30,080 at Empreinte Digital and who is actually the person who manages all the streams that we 5 00:00:30,080 --> 00:00:36,520 have today and that we had yesterday in Matomo Camp. Just to say a few words about Valentin, 6 00:00:36,520 --> 00:00:43,480 so Valentin is DevOps at Empreinte Digital which is a company located in Angers. His main 7 00:00:43,480 --> 00:00:50,120 mission is the deployment and maintenance of the cloud structure in SAS mode of the Empreinte 8 00:00:50,120 --> 00:00:56,040 Digital which is a data center that is spread between Angers, Rennes and Tours. In this 9 00:00:56,040 --> 00:01:02,960 conference, Valentin will tell us how we manage a Matomo instance on a daily basis, 10 00:01:02,960 --> 00:01:08,160 whether it is an instance that is very simple, that does not have a lot of traffic, 11 00:01:08,160 --> 00:01:15,400 up to an instance that has up to 5 million visitors per month. So it is above all a 12 00:01:15,400 --> 00:01:22,120 conference in experience return mode for nearly two years. Empreinte Digital and Valentin 13 00:01:22,120 --> 00:01:28,440 are currently managing a client that some of you may know, which is PIX. PIX is 14 00:01:28,440 --> 00:01:33,960 actually a platform that allows you to test digital knowledge which is notably used 15 00:01:33,960 --> 00:01:40,600 in the world of higher education. It is a little bit the equivalent of TOEIC but 16 00:01:40,600 --> 00:01:45,600 in digital culture mode. So it's a little bit like how we could see it in the 17 00:01:45,600 --> 00:01:52,680 conference yesterday hosted by Roré from the Digitalist AB team. We are on a site 18 00:01:52,680 --> 00:01:58,240 that receives a lot of traffic and which manages to keep up with a Matomo who is 19 00:01:58,240 --> 00:02:04,840 there to monitor all of this. That's it. Listen, I'm going to leave the floor to Valentin. Do not hesitate 20 00:02:04,840 --> 00:02:10,840 during the whole conference to interact, to ask questions on the platform which I remind you is 21 00:02:10,840 --> 00:02:17,560 chat.matomocamp.org on which you can register. You have a room that corresponds 22 00:02:17,560 --> 00:02:22,140 to the conference we currently have. You ask all your questions there. And if you 23 00:02:22,140 --> 00:02:27,640 want to continue to ask questions to Valentin, do not hesitate to use this chat 24 00:02:27,640 --> 00:02:37,040 which will be active for the whole day. I wish you all a very good conference. 25 00:02:37,040 --> 00:02:42,480 Hello everyone. So first of all I'm going to try to fix a little problem at the 26 00:02:42,480 --> 00:02:49,960 level of the conference because a priori it does not appear. So I'm going to allow myself to 27 00:02:49,960 --> 00:02:59,240 upload the conference to you. These are the little direct alerts we are going to say. 28 00:03:10,920 --> 00:03:11,320 So 29 00:03:11,320 --> 00:03:22,360 I'm sorry for this little setback. 30 00:03:41,320 --> 00:04:11,160 Downloaded. So hello everyone. 31 00:04:12,120 --> 00:04:18,720 My name is Valentin. Thank you Renan for giving me the floor to talk to you about what is a 32 00:04:18,720 --> 00:04:25,880 upload of Matomo. So we're going to talk about how it is possible to create a Matomo 33 00:04:25,880 --> 00:04:32,640 instance easily. To show how easy it is to set up and maintain a software. 34 00:04:32,640 --> 00:04:39,920 And also to see how it is possible to upload up to loads that we 35 00:04:39,920 --> 00:04:44,480 were able to meet yesterday in the first day up to 50 million visits per month, etc. 36 00:04:44,480 --> 00:04:50,880 So first of all, who am I? My name is Valentin Barret. I am a DevOps Engineer at 37 00:04:50,880 --> 00:05:00,000 Empreinte Digital. We have been running Matomo for a little over two years. We started 38 00:05:00,000 --> 00:05:08,240 little by little on this software by obviously appetizing to open source and to 39 00:05:08,240 --> 00:05:14,080 offer additional service to our customers. And so for two years we have acquired a lot of 40 00:05:14,080 --> 00:05:19,200 experience until we will see, we will reach rather interesting numbers on the 41 00:05:19,200 --> 00:05:28,520 management of Matomo instances. So do I have to introduce Matomo? Well, not really, 42 00:05:28,520 --> 00:05:33,880 we are at MatomoComp, but I still wanted to say a few things. It was that Matomo, 43 00:05:33,880 --> 00:05:39,040 it is formerly Peweak, that it has existed for a very long time. It is an open source software 44 00:05:39,040 --> 00:05:45,480 for web statistics, successor to PHP MyVisit and designed to be an alternative 45 00:05:45,480 --> 00:05:50,520 free to Google Analytics, Matomo works on PHP and MySQL servers. And as Renan 46 00:05:50,520 --> 00:05:56,120 said very precisely yesterday, you should know that Matomo is neither more nor less than a database 47 00:05:56,120 --> 00:06:00,240 with data that we collect as we go. And what is also interesting is to 48 00:06:00,240 --> 00:06:03,960 say that Matomo is a software that has existed for years and years and that it is based 49 00:06:03,960 --> 00:06:12,640 on essentially two bricks, PHP and MySQL. So it's only two simple components 50 00:06:12,640 --> 00:06:18,800 that allow you to have a tool as powerful as Matomo. PHP which will allow you to process 51 00:06:18,800 --> 00:06:24,280 information, so obviously there is a bit of language, now there is JavaScript, there is PHP, 52 00:06:24,280 --> 00:06:28,160 there is a little more tuning around all that, but originally you have to know that it is really 53 00:06:28,160 --> 00:06:33,360 two technologies, the simplest possible to create analytics tools that we know today. 54 00:06:33,360 --> 00:06:41,400 So the first approach with Matomo. So I have my little images that do not work 55 00:06:41,400 --> 00:07:09,080 because of the PDF rendering, but it does not matter. The first approach with Matomo. So I just want to tell you the story of the 56 00:07:09,080 --> 00:07:13,280 implementation of Matomo as we could hear it today. That is to say that I place myself 57 00:07:13,280 --> 00:07:18,560 as a dev, I am not a dev, I am a sysops with a dev habitat, but I place myself 58 00:07:18,560 --> 00:07:21,760 as a dev, I have an application, I want to test the audience, Google I do not like it too much, 59 00:07:21,760 --> 00:07:29,840 I know that there are some privacy problems, I have RGPD, etc. So I tell myself, 60 00:07:29,840 --> 00:07:34,920 I type on Google, I easily come across Matomo and I tell myself that it seems to correspond to my needs, 61 00:07:34,920 --> 00:07:40,560 in addition, I looked at the list of integrations, I can integrate it very easily in WordPress, 62 00:07:40,560 --> 00:07:45,200 in Hugo, etc. There are more than 40 integrations that have been created by the community. 63 00:07:45,200 --> 00:07:53,440 So I tell myself, but ok, perfect, I'm going to put a piece in this software. 64 00:07:53,440 --> 00:08:01,480 And so there, what I can see is that Matomo can track up to more than 100,000 pages, 65 00:08:01,480 --> 00:08:08,680 so page views per month, with, let's say, quite a few resources, that is to say 2 CPUs, 66 00:08:08,680 --> 00:08:16,080 2 GB of RAM and 50 GB of SSD for the disk in particular. So that means that on my application 67 00:08:16,080 --> 00:08:23,160 that I put at my disposal for my development, which aims to sell, I am a dev, 68 00:08:23,160 --> 00:08:27,760 I created a product, I want to sell it, I want to make it known, etc. And I want to see 69 00:08:27,760 --> 00:08:35,000 how the users use it. And well, quite simply, I will be able to track, 70 00:08:35,000 --> 00:08:41,840 let's say, the user on the functioning of my website. And all this without changing hosting, 71 00:08:41,840 --> 00:08:49,280 without adding CPU cores, because I took a forfeit at OVH, at Scaleway, at Emprunt Digital, 72 00:08:49,280 --> 00:08:59,640 and it is possible to have my hosting next to my Matomo. So it's easy to install, 73 00:08:59,640 --> 00:09:06,120 I have my integrations, easy to maintain, it's PHP with a database, there may be 74 00:09:06,120 --> 00:09:12,080 some surprises, but overall all the updates are going pretty well. It corresponds to all my 75 00:09:12,080 --> 00:09:15,520 needs. My goal at the beginning is to really have my page views, to know how many people 76 00:09:15,520 --> 00:09:22,120 access my application and how many pages of my application have been consulted, etc. 77 00:09:22,120 --> 00:09:27,400 And it works automatically, that is to say that once we have installed the Matomo, 78 00:09:27,400 --> 00:09:32,400 once we have put the tracking, the data almost easily enter the application 79 00:09:32,400 --> 00:09:40,120 and it is possible, in addition to that, to consult them so easily. So there, 80 00:09:40,120 --> 00:09:48,840 the image is missing, it doesn't matter, I'm talking about 100 to 1 million users, 81 00:09:48,840 --> 00:09:52,880 but for now you offer us to have an installation for a few users, because I remind you 82 00:09:52,880 --> 00:10:02,080 that here, my goal at the beginning was to create my application, to have tracking, 83 00:10:02,080 --> 00:10:07,600 but I know that when I start a business, unless I have been driven by big guys, 84 00:10:07,600 --> 00:10:14,960 I will only have a few users who will, how to say, consult my application at the beginning. 85 00:10:14,960 --> 00:10:22,080 So you tell us, I'm going to go from 100 users to more than a million, 86 00:10:22,080 --> 00:10:28,840 but how are we going to do it because we are on an installation that is 2 CPUs, 87 00:10:28,840 --> 00:10:35,320 2 GB of RAM, and in addition to that you show us that the requirements can go only up to 88 00:10:35,320 --> 00:10:43,400 100,000 pageviews per month. No, no, I'm not fooling you, don't worry, 89 00:10:43,400 --> 00:10:49,360 my application works well, we want more visitors, I have more visitors, 90 00:10:49,360 --> 00:10:56,720 my business is developing, it's great, I go from me alone in my garage to a little more 91 00:10:56,720 --> 00:11:03,120 in my company that I created in order to develop my product, and so I have a team 92 00:11:03,120 --> 00:11:11,640 around that in the company. We are now 3-4 devs, I had a web analyst 93 00:11:11,640 --> 00:11:20,040 enter my company because we know that it is also important today to know the 94 00:11:20,040 --> 00:11:27,760 audience, especially on products in extra web. And so suddenly the web analyst tells me 95 00:11:27,760 --> 00:11:32,240 oh there we are starting to have a lot of traffic, but on the other hand there is a latency for my access to my 96 00:11:32,240 --> 00:11:39,120 Matomo. So a little aside, I'm talking to you about a feedback, it's pretty much 97 00:11:39,120 --> 00:11:43,120 what we had, that is to say that more than two years ago, it's been a long time since we used Matomo, 98 00:11:43,120 --> 00:11:48,080 but it's been more than two years now that we're trying to promote it, also for our 99 00:11:48,080 --> 00:11:57,160 clients I mean. We were using Matomo and we realized that damn the access now to the 100 00:11:57,160 --> 00:12:00,840 websites could be a little slower by a few milliseconds, but the web analyst 101 00:12:00,840 --> 00:12:06,480 told us yeah it's still a bit embarrassing because it slows down almost the entire stack 102 00:12:06,480 --> 00:12:14,320 just because we added tracking. So there is a pretty simple tip to improve 103 00:12:14,320 --> 00:12:20,400 the experience in terms of access to tracking and therefore easier recording. 104 00:12:20,400 --> 00:12:27,000 So we talked about it yesterday with the first talks of Tips and Tricks, and then 105 00:12:27,000 --> 00:12:32,160 globally all the people who talked about infrastructure. The must-have in this case, 106 00:12:32,160 --> 00:12:38,760 and in almost all Matomo installations, is obviously the Qt tracking. 107 00:12:38,760 --> 00:12:48,240 What is Qt tracking? Well, it's the tailing of the statistics, how to say, 108 00:12:48,240 --> 00:12:54,840 of the user's follow-up code in a queue. So here we are talking about Redis or MySQL, 109 00:12:54,840 --> 00:13:04,320 which will allow us to wait for the records of the website and to ingest them, 110 00:13:04,320 --> 00:13:10,160 to feed them by MySQL in batches, that is to say that rather than putting them one by one 111 00:13:10,160 --> 00:13:15,800 as we go along and having a complete follow-up of the request until it is stored, 112 00:13:15,800 --> 00:13:20,400 here we are going to put the different requests in a box and once this box is filled, 113 00:13:20,400 --> 00:13:30,200 we put it in, how to say, Matomo's PHP, which will say, oh look, the box is filled, 114 00:13:30,200 --> 00:13:35,240 I'm going to put this box directly to the database. And that, by the way, greatly improves 115 00:13:35,240 --> 00:13:43,200 the performance. So that's what I'm saying, the principle is simple, a waiting line 116 00:13:43,200 --> 00:13:46,680 to record more quickly the accesses and reduce the load in a tight flow. More info, 117 00:13:46,680 --> 00:13:51,200 I put the link of the plugin QtTracking, do not hesitate to go and check it out, 118 00:13:51,200 --> 00:13:55,600 it is really very interesting and we have been able to see it over the different conferences, 119 00:13:55,600 --> 00:14:04,800 it is a must-have. So it's great, I have a demo, I'll try to do it, 120 00:14:04,800 --> 00:14:12,960 I hope it works, otherwise I'll skip it and then it's not very fun. So it's great, 121 00:14:12,960 --> 00:14:20,520 the web analyst tells me, great, thanks to your little optimizations, you were able to 122 00:14:20,520 --> 00:14:28,360 increase the score of the page load, etc. And on top of that, we didn't need to 123 00:14:28,360 --> 00:14:35,880 take up an additional hosting, oh damn, I'm sorry, the diagram doesn't work, 124 00:14:35,880 --> 00:14:47,760 it doesn't matter, I'll do it for you. We will say that we have the users here, we have 125 00:14:47,760 --> 00:15:17,200 the matomo here, finally the inframatomo, we improvise, it doesn't matter. And so, where before I had my matomo with my database, 126 00:15:17,200 --> 00:15:35,840 I could, so I'm just going to make a little note here, and add here. So the users 127 00:15:35,840 --> 00:15:40,400 arrive on matomo and there is a connection between the database and matomo, we were able to add 128 00:15:40,400 --> 00:15:49,800 here what we call QtTracking via a Redis database. And so from an infrastructure point of view, 129 00:15:49,800 --> 00:15:56,280 it's very simple, we have the user who will record the requests in the matomo, 130 00:15:56,280 --> 00:16:01,200 and rather than going from matomo to the database, we will go through the Redis which will allow us 131 00:16:01,200 --> 00:16:07,240 to stamp all of these requests and therefore to avoid overloading the PHP used by matomo 132 00:16:07,240 --> 00:16:12,280 and also by the application, we can imagine that it is a Wordpress, a Drupal, any PHP application, 133 00:16:12,280 --> 00:16:17,560 and therefore that the PHP is shared. And so rather than overloading the PHP of the entire machine, 134 00:16:17,560 --> 00:16:23,800 so the small VPS that we can imagine, we will just pass the plates in the Redis and matomo 135 00:16:23,800 --> 00:16:31,600 will act, we will say at an almost regular interval, to launch the integration of the requests 136 00:16:31,600 --> 00:16:38,720 in the database. So we have a very simple infrastructure, we can scale, so we went from 137 00:16:38,720 --> 00:16:48,160 a few users to, we will say, we can perhaps imagine several thousand users in the infrastructure 138 00:16:48,160 --> 00:16:57,200 without changing much. QtTracking is a really important asset in matomo. And that's 139 00:16:57,200 --> 00:17:03,840 great, that is to say that even if we have our business that is increasing, that we have more and more 140 00:17:03,840 --> 00:17:14,280 users, we have a small plugin to install and let's go, we can delay the increase 141 00:17:14,280 --> 00:17:22,360 of the infrastructure. So the demo, I'm not going to do it, we'll do it at the end if I have time 142 00:17:22,360 --> 00:17:33,320 to really show you how effective it is. In theory, the demo is a success. 143 00:17:33,320 --> 00:17:38,800 And so I ask myself the question of how to make sure of the point of break. Obviously there I 144 00:17:38,800 --> 00:17:47,120 told you, the web analyst told us, here I would like to increase the response time of 145 00:17:47,120 --> 00:17:56,680 access to tracking, so small JS or small PHP request, etc. for matomo. But there is another 146 00:17:56,680 --> 00:18:02,520 problem, that is to say that obviously we delay the increase of the infrastructure. I remember 147 00:18:02,520 --> 00:18:09,640 earlier we were on two 2 GB RAM CPUs, 50 SSDs. Come on, we said to ourselves, we went from 4 GB RAM CPUs, 148 00:18:09,640 --> 00:18:16,040 great, we were able to increase the load even more, but now I'm starting to have a lot of 149 00:18:16,040 --> 00:18:26,160 slowdowns on my application, as much as cms, imagine it's a cms, than for the integration of 150 00:18:26,160 --> 00:18:31,800 data in matomo. And on top of that, I have the impression that sometimes it's not quite representative 151 00:18:31,800 --> 00:18:39,560 of what can happen. So how to detect problems? So obviously the first problem 152 00:18:39,560 --> 00:18:45,880 which is quite simple to identify is the slow access. We have trouble accessing the application, 153 00:18:45,880 --> 00:18:55,360 rather than loading the application in 1 second, 500 milliseconds, etc. We find ourselves at 3-4 154 00:18:55,360 --> 00:19:01,120 seconds, we have errors 500 in server and in addition we have timeouts that appear in the 155 00:19:01,120 --> 00:19:10,080 vhost, in the matomo application. So small aside, second small aside, I'm talking about 156 00:19:10,080 --> 00:19:17,200 concepts, I'm not talking about techno. All this can be done in barmetal, in container, in sass, 157 00:19:17,200 --> 00:19:20,960 knowing that the sass obviously it's not really you who manage, but we can still 158 00:19:20,960 --> 00:19:26,760 realize small slow access and small timeouts, even errors 500 if we have monitoring etc. 159 00:19:26,760 --> 00:19:32,760 In short, I'm talking about concepts and there is no techno behind all this, you do it as you 160 00:19:32,760 --> 00:19:38,240 want. This is also the advantage of this matomo application, it's PHP, so it's 161 00:19:38,240 --> 00:19:44,560 relatively standard, so you do it roughly as you want. So how to detect problems? 162 00:19:44,560 --> 00:19:48,800 These small indicators can tell us, ah it's weird, that's it, my matomo I have the impression that it is 163 00:19:48,800 --> 00:19:54,600 a little overloaded. So how can we increase the load? So I said a little bit 164 00:19:54,600 --> 00:19:57,560 earlier, we can increase the cpu, we can increase the memory and hop it starts again, 165 00:19:57,560 --> 00:20:02,320 indeed we can increase the cpu, we can increase the memory and it starts again, 166 00:20:02,320 --> 00:20:10,360 we will say almost. It's not that complicated to increase the load, but we can't increase 167 00:20:10,360 --> 00:20:17,840 infinitely, that is to say that at one point we are not going to order machines with 128 cpu, 168 00:20:17,840 --> 00:20:24,680 3 terabytes of ram, 6 terabytes of storage, just for the pleasure of our eyes, we will say, and to 169 00:20:24,680 --> 00:20:28,680 say to ourselves, it's good, I'm going to manage my matomo infrastructure. No, it can't work like that. 170 00:20:28,680 --> 00:20:36,240 For what reason? Simply high availability. Our business, we started from a developer 171 00:20:36,240 --> 00:20:42,320 to a team of developers and now that's it, I feel like I'm well into the business and 172 00:20:42,320 --> 00:20:50,240 then it's really part of the identity now, the analytics, I offer products with 173 00:20:50,240 --> 00:20:54,360 analytics etc. So it really has to be that this matomo infrastructure is a key point 174 00:20:54,360 --> 00:21:00,720 and an additional motivation to ensure it in the infrastructure of my company. And 175 00:21:00,720 --> 00:21:05,600 so for that, you obviously have to have high availability. High availability goes through 176 00:21:05,600 --> 00:21:14,800 what? Well, through a lot of things that we will see just after. So can we scale horizontally? 177 00:21:14,800 --> 00:21:23,200 I'm sorry, every time there are small diagrams but I will provide them to you after with the 178 00:21:23,200 --> 00:21:29,040 PDF better generated. So can we scale horizontally? In high availability, 179 00:21:29,040 --> 00:21:32,120 we can ask ourselves this question. Can we scale horizontally? Yes we can. 180 00:21:32,120 --> 00:21:37,920 What components? The PHP application, the Redis, the database and of course we don't forget, 181 00:21:37,920 --> 00:21:43,320 last but not least, a LB. LB for calls, for technical names or for techniques, 182 00:21:43,320 --> 00:21:48,400 the load balancer, so to manage the load on the different PHP applications. 183 00:21:48,400 --> 00:21:55,640 The advantage is that it is an application that is in PHP, which has different components, 184 00:21:55,640 --> 00:22:01,040 which can work in different ways. We have seen it with the different conferences, 185 00:22:01,040 --> 00:22:06,680 especially with the conference of our Italian friends, which was great, 186 00:22:06,680 --> 00:22:12,760 I really invite you to go and see it. They have really separated each PHP component 187 00:22:12,760 --> 00:22:22,000 with the cron, the task scheduler, the tracking, etc. So great, we know that Matomo, 188 00:22:22,000 --> 00:22:27,360 we can scale it. That's good news. That is to say that from now on, we are able to 189 00:22:27,360 --> 00:22:33,680 switch from a small infrastructure on a server, a database, to different servers, 190 00:22:33,680 --> 00:22:39,920 a Redis database that can be clustered, several databases that can do read-write, etc. 191 00:22:39,920 --> 00:22:49,120 In short, a pleasure. And here we don't have the infrastructure. So I'm going to 192 00:22:49,120 --> 00:22:59,960 allow myself to add a small image because I find this one rather interesting. So I 193 00:22:59,960 --> 00:23:13,600 ask you two minutes. On the page to manage the infrastructure, it is also to know the tools 194 00:23:13,600 --> 00:23:18,400 for the Matomo camp. So I can afford to offer you the possibility to see what I 195 00:23:18,400 --> 00:23:23,120 wanted to show you. And so now we end up with an infrastructure that is more complex. 196 00:23:23,120 --> 00:23:27,400 So this is the visualization of what I was explaining to you just before. We have the 197 00:23:27,400 --> 00:23:31,520 requests that arrive, we have a load balancer that is distributed on the different Matomo. So it 198 00:23:31,520 --> 00:23:38,680 can be full applications or not, as I said at the moment, with different microservices, 199 00:23:38,680 --> 00:23:46,400 which uses a Redis cluster or a Redis node. In short, there are plenty of solutions. And then 200 00:23:46,400 --> 00:23:53,520 we have a MySQL database that is in replication, master-master, master-slave. In short, 201 00:23:53,520 --> 00:23:59,160 obviously, we have plenty of imagination to improve the infrastructure. Obviously, 202 00:23:59,160 --> 00:24:07,240 it is also limited by the choices that have been made by Matomo. We talked about it yesterday 203 00:24:07,240 --> 00:24:13,040 with the different databases that would like to be integrated into Matomo. This is 204 00:24:13,040 --> 00:24:21,040 the result of the MySQL that I took, obviously. Success! In relation to our little installation, 205 00:24:21,040 --> 00:24:26,120 we asked ourselves questions and we realized now that it was possible to have 206 00:24:26,120 --> 00:24:33,080 a Matomo infrastructure of highly available and scalable production. Great, 207 00:24:33,080 --> 00:24:41,120 we went from 100,000 to more than 100 million visits per month. It's not me who said it, 208 00:24:41,120 --> 00:24:46,880 it's obviously the Matomo requirements page. And I schematize a little because in fact, 209 00:24:46,880 --> 00:24:51,200 once we understood how it was possible to add this or that component, 210 00:24:51,200 --> 00:24:57,240 of what is this or that component, it is easy to... I say easy because we will see 211 00:24:57,240 --> 00:25:02,520 that there are small subtleties, but it is relatively easy to understand how we can 212 00:25:02,520 --> 00:25:09,040 increase the load of our Matomo. And I didn't talk about it, but we can even, 213 00:25:09,040 --> 00:25:13,880 once we are comfortable with the infrastructure, we are comfortable with the different elements 214 00:25:13,880 --> 00:25:19,680 that we have put at the level of tracking, etc. Put our little JavaScript code on a 215 00:25:19,680 --> 00:25:31,120 CDN to further increase the possibility of quick access to different points of the globe. 216 00:25:31,120 --> 00:25:38,640 I didn't show you that. Obviously, all of this can be done in bar-metal clusters, 217 00:25:38,640 --> 00:25:44,760 in Kubernetes clusters, in SWAM clusters. I remind you, I'm not talking about techno, 218 00:25:44,760 --> 00:25:49,520 I'm talking about concepts. Here we are on things that allow us to understand how 219 00:25:49,520 --> 00:25:54,000 it is easy to go from 100 to several million users per month. 220 00:25:54,000 --> 00:26:02,360 Well, a little surprise that we had in our experience is that the web analyst 221 00:26:02,360 --> 00:26:05,840 wants to use a tag manager because the goal is to have even more ways to analyze the 222 00:26:05,840 --> 00:26:10,200 use of the site. Great! Renan will tell you about it, tag manager is a great tool, 223 00:26:10,200 --> 00:26:16,480 Renan Chardonneau and Renan Hélo, I remember, who talk about it regularly. These are great 224 00:26:16,480 --> 00:26:25,320 tools, but we realized one thing, it is that in a clustered environment, 225 00:26:25,320 --> 00:26:30,560 the tag manager is not really designed to be clustered. So there are issues 226 00:26:30,560 --> 00:26:38,440 in this sense. We, in our case of client experience and return of experience, 227 00:26:38,440 --> 00:26:43,360 had to find a solution relatively quickly. So how did we figure it out? 228 00:26:43,360 --> 00:26:48,360 What does a tag manager mean? In fact, the tag manager cannot be 229 00:26:48,360 --> 00:26:52,800 clustered, it cannot be used on all nodes. In fact, the tag manager is generated 230 00:26:52,800 --> 00:26:57,960 locally on an PHP instance, which means that it depends on which worker it is generated 231 00:26:57,960 --> 00:27:03,480 when an administrator uses the platform. So, quite simply, when an update 232 00:27:03,480 --> 00:27:08,040 of the tag manager is done, if there is not something that allows you to link 233 00:27:08,040 --> 00:27:14,200 the different... so here we are talking about javascript, the different javascripts 234 00:27:14,200 --> 00:27:19,000 so that it is common between the different workers, there can be inconsistencies, 235 00:27:19,000 --> 00:27:23,200 that is to say that a client, a user will arrive on version 1 of the tag manager while 236 00:27:23,200 --> 00:27:29,400 the other will arrive on a version 2. It is not really the spirit that we want to have 237 00:27:29,400 --> 00:27:38,400 and the behavior that we want to have. So for that, we had to... well, 238 00:27:38,400 --> 00:27:42,680 we can ask the question of does that mean that the tag manager is not 239 00:27:42,680 --> 00:27:46,360 clustered on the most complex platforms with different nodes? So that's not what 240 00:27:46,360 --> 00:27:50,000 I mean. No, no, we can do it, of course, but there are a few tricks to achieve 241 00:27:50,000 --> 00:28:05,440 the expected result. So, here I am talking about that. We could use a shared 242 00:28:05,440 --> 00:28:10,640 file system between the different workers to have exactly the same code. So what 243 00:28:10,640 --> 00:28:14,240 Mattomo recommends is that you have to synchronize when you are in a clustered environment 244 00:28:14,240 --> 00:28:21,600 the PHP configuration. So this is typically the, we will say, base of knowledge 245 00:28:21,600 --> 00:28:26,040 of the configuration of Mattomo and upload all the files and plugins 246 00:28:26,040 --> 00:28:31,600 on each server. Okay, so actually, you need to have about the same base. 247 00:28:31,600 --> 00:28:40,360 How can we make sure that there is always the same codebase etc. and on top of that, 248 00:28:40,360 --> 00:28:46,600 reduce the perfs, avoid using an NFS or a shared file. I like hypotheses 249 00:28:46,600 --> 00:28:50,600 or even an S3, etc. I don't want to lose too much performance because obviously 250 00:28:50,600 --> 00:28:59,240 my application is clustered, etc. So suddenly, we say to ourselves, we have to synchronize everything, 251 00:28:59,240 --> 00:29:05,400 but in the base of Mattomo, there is also the TMP and in the TMP, it is there that 252 00:29:05,400 --> 00:29:12,360 the tag manager is generated, the tag manager JavaScript. And then we realize that, is 253 00:29:12,360 --> 00:29:16,680 it really important to generate the TMP because it also contains a lot of things, 254 00:29:16,680 --> 00:29:24,000 cache, etc. Does that mean that it is a file that is very regularly 255 00:29:24,000 --> 00:29:30,240 written, with potentially a lot of access, etc. So if we put a shared file system, 256 00:29:30,240 --> 00:29:37,840 this shared file system must be very efficient. So there, the same, to 257 00:29:37,840 --> 00:29:44,360 redo the little aside, I'm talking about concepts and I say to myself, at the beginning, I don't want to 258 00:29:44,360 --> 00:29:48,280 spend thousands of cents on super complex file systems, etc. so that it works. 259 00:29:48,280 --> 00:29:57,000 So what can I do? So there, I'll do it for you. What can I do? 260 00:29:57,000 --> 00:30:03,640 I have no solution, my tag manager is generated aside and if I want to update it, 261 00:30:03,640 --> 00:30:08,360 I have to go to the machines, that I copy it, finally I don't know how to do it. So we, 262 00:30:08,360 --> 00:30:12,240 the solution that we found, in any case in the emergency, finally in the emergency, 263 00:30:12,240 --> 00:30:18,920 not in the emergency but in our case that we had, was to use SingSing. So SingSing, 264 00:30:18,920 --> 00:30:22,920 I don't know if you know what it is, but it is a small tool that allows you to synchronize 265 00:30:22,920 --> 00:30:28,720 different repertoires and in addition to that there is a management of the different versions of 266 00:30:28,720 --> 00:30:33,720 files that have been synchronized. It's not live, it's in sync, but be careful, 267 00:30:33,720 --> 00:30:40,520 we're talking about in sync in a few milliseconds. And above all, it avoids having 268 00:30:40,520 --> 00:30:47,400 locks or a file system that is dependent on another machine and in addition to that, 269 00:30:47,400 --> 00:30:52,040 avoiding concurrent accesses at the file system level. In short, we synchronized the tags. Great, 270 00:30:52,040 --> 00:30:57,880 that is to say that we have our tag manager which is now available on all of our workers 271 00:30:57,880 --> 00:31:05,840 and which allows web analysts who connect to the platform to take care of creating 272 00:31:05,840 --> 00:31:11,840 the different tags etc. and to put them into production instantly. So obviously, 273 00:31:11,840 --> 00:31:16,280 I'm talking about our case, it's a REX, we used SingSing, there are lots of other solutions, 274 00:31:16,280 --> 00:31:24,600 it's up to you to do whatever you want. I found it interesting to talk about this little case 275 00:31:24,600 --> 00:31:32,120 of the tag manager which is now almost an essential part of the use of Matomo 276 00:31:32,120 --> 00:31:38,560 and which suddenly caused us some problems in our Matomo administration. So let's 277 00:31:38,560 --> 00:31:44,720 make a point. I remind you that at the beginning we had our little dev who was alone on his project 278 00:31:44,720 --> 00:31:54,920 who started to have his business which increased and suddenly he hired people, he hired 279 00:31:54,920 --> 00:32:01,040 admins, he hired devops. In short, it's good, it's a company that is well oiled, 280 00:32:01,040 --> 00:32:09,640 that uses Matomo at scale etc. And all that, I didn't tell you, but obviously these are 281 00:32:09,640 --> 00:32:14,880 maintenance that can be done almost without downtime. I remind you that a database, 282 00:32:14,880 --> 00:32:21,560 it is possible to put it into replication easily, it is possible to create a new Matomo worker, 283 00:32:21,560 --> 00:32:28,000 to put in front a load balancer. So suddenly, the time to stop a Matomo on our 284 00:32:28,000 --> 00:32:36,240 tiny 2GB RAM CPU, I redirect the traffic. We managed to scale the entire application, 285 00:32:36,240 --> 00:32:40,520 to go from 100 users to more than a million, almost without downtime, by asking ourselves the 286 00:32:40,520 --> 00:32:45,440 right questions, by having had a little feedback, by realizing that finally 287 00:32:45,440 --> 00:32:48,680 the Matomo application was sufficiently well done to be able to scale it horizontally. 288 00:32:48,680 --> 00:32:54,080 And especially by adding the Qt tracking which is a very important component in terms of 289 00:32:54,080 --> 00:32:58,560 performance but also in terms of maintenance. I didn't talk about it obviously but the Qt tracking 290 00:32:58,560 --> 00:33:05,240 also allows during maintenance to continue to recover the data. So it will 291 00:33:05,240 --> 00:33:10,560 store them, store them, store them. And once the maintenance is finished, it will take them, 292 00:33:10,560 --> 00:33:16,160 send them to the database and it will resume the course of its life, we will say, to store 293 00:33:16,160 --> 00:33:22,680 the requests as and when needed. So great, but finally in summary, what did we see? 294 00:33:22,680 --> 00:33:31,080 Well in summary, we saw that Qt tracking is a must-have. So I will answer questions 295 00:33:31,080 --> 00:33:35,280 after but I saw a very small one from Ronan who was talking about the fact that Matomo is 296 00:33:35,280 --> 00:33:40,640 important even on small instances. For me yes it is very important because it improves 297 00:33:40,640 --> 00:33:46,120 a lot of things and it allows you to absorb traffic peaks. Can we scale easily? 298 00:33:46,120 --> 00:33:52,520 Yes we can scale easily. We saw it in the concepts, it's easy. There are even 299 00:33:52,520 --> 00:33:57,720 little tips for the tag manager when we have problems. Obviously there are always 300 00:33:57,720 --> 00:34:04,880 some concerns about the load. We agree on that. I invite you to go 301 00:34:04,880 --> 00:34:12,640 see yesterday's conferences. There were several who talked about these little problems, 302 00:34:12,640 --> 00:34:17,960 we will say. And really they made very enriching feedback also on the 303 00:34:17,960 --> 00:34:29,000 management and the scale of Matomo. So here I am. I will allow myself to show you 304 00:34:29,000 --> 00:34:39,840 the slide, the little photo because normally I have a small image that is displayed. 305 00:34:39,840 --> 00:34:53,080 We are in summary. We can go from a few hundred users per month to more than 306 00:34:53,080 --> 00:35:02,680 5 million users. This is really a concrete case. It is thanks to our experience 307 00:35:02,680 --> 00:35:11,560 of getting in charge that we get to this result. We reached more than 5 million 308 00:35:11,560 --> 00:35:16,720 visits, more than 100 million views, more than 131 million shares. Obviously if you 309 00:35:16,720 --> 00:35:20,280 are not necessarily familiar with Matomo, it may not be worth talking to you. But 310 00:35:20,280 --> 00:35:26,200 these are figures that are quite impressive and on which we are rather satisfied 311 00:35:26,200 --> 00:35:32,200 within the digital footprint team on our use and our getting in charge with Matomo. 312 00:35:32,200 --> 00:35:39,680 Do we have anything else to say? Yes, because there we talked about getting in charge, 313 00:35:39,680 --> 00:35:44,520 concepts, etc. The fact that we were not really dependent on Techno, just 314 00:35:44,520 --> 00:35:50,040 the functioning of the architecture. We did not say everything. That is to say that we 315 00:35:50,040 --> 00:35:55,800 still have a few small problems that can arise and that is in the generation of 316 00:35:55,800 --> 00:36:03,880 reports. We still encounter a few problems with some customers. The generation 317 00:36:03,880 --> 00:36:10,600 of reports takes a lot of time. These reports are generated by the account, etc. 318 00:36:10,600 --> 00:36:16,080 So there are still optimizations to be made. It should not be forgotten that it is important 319 00:36:16,080 --> 00:36:21,600 to increase beyond the Matomo application, the applications on which Matomo rests, 320 00:36:21,600 --> 00:36:27,400 the timeouts of PHP, Nginx, Proxy, etc. to prevent that when there are generations 321 00:36:27,400 --> 00:36:36,880 of reports, in particular the PHP is killed. There are optimizations of the base 322 00:36:36,880 --> 00:36:41,440 of the data to be made. There is the optimization of Quick Tracking. I'm talking about the optimization 323 00:36:41,440 --> 00:36:49,920 of Quick Tracking because we had a little trouble getting in charge when we had 324 00:36:49,920 --> 00:36:53,680 a peak at one point. That is to say that the Quick Tracking worked correctly but we did not have 325 00:36:53,680 --> 00:37:04,640 a view in the famous real-time. There is a small real-time that allows you to know 326 00:37:04,640 --> 00:37:12,120 the number of users who are currently on the platform. In fact, it was still 327 00:37:12,120 --> 00:37:16,520 at 0 and it was because the Quick Tracking was increasing, increasing, increasing, increasing, 328 00:37:16,520 --> 00:37:22,800 until the PHP could no longer take the different loads and send it to the database. 329 00:37:22,800 --> 00:37:29,200 So by optimizing a little bit, we were able to achieve more than 9,000 requests 330 00:37:29,200 --> 00:37:33,400 in the last three minutes. So in the real-time visitor count, you will see, 331 00:37:33,400 --> 00:37:39,760 more than 9,000 users in real time on the platform and for our greatest happiness. 332 00:37:39,760 --> 00:37:45,920 And as our Italian friend said, he was looking at this little real-time count, 333 00:37:45,920 --> 00:37:47,920 I was the same to check that everything was working correctly. 334 00:37:47,920 --> 00:37:56,280 That's it, I'm done. I hope it was instructive. I hope it was, let's say, 335 00:37:56,280 --> 00:38:06,320 not too complicated in terms of management concepts, etc. My goal was to see 336 00:38:06,320 --> 00:38:11,240 general concepts to explain to you that finally the Matomo application was 337 00:38:11,240 --> 00:38:16,200 well made enough to scale it and that we had little limit to get 338 00:38:16,200 --> 00:38:22,560 on board with the application. That's it. So now I allow myself to go 339 00:38:22,560 --> 00:38:31,720 to the chat to consult the different questions. So Renan who asks us, 340 00:38:31,720 --> 00:38:35,680 should we put the Quick Tracking even on the infrastructure, in case we would have 341 00:38:35,680 --> 00:38:42,240 a traffic peak? Indeed, the Quick Tracking, I recommend it de facto 342 00:38:42,240 --> 00:38:48,400 on any type of installation because it is important to be able to absorb 343 00:38:48,400 --> 00:38:52,120 these small traffic peaks. And if in addition to that we have an application, 344 00:38:52,120 --> 00:38:59,560 so if we have our Matomo with our PHP application, how to say, 345 00:38:59,560 --> 00:39:03,520 the Quick Tracking will allow us to avoid overloading the entire machine. 346 00:39:03,520 --> 00:39:09,200 So I think it's important to put it. Knowing that we don't have to put 347 00:39:09,200 --> 00:39:15,600 Redis with the Quick Tracking, even if I think it's not too consumerist to put Redis, 348 00:39:15,600 --> 00:39:19,120 we can use the Quick Tracking in MySQL directly. 349 00:39:19,120 --> 00:39:25,720 Have you had to make some modifications to improve the archiving? 350 00:39:25,720 --> 00:39:36,240 So we tried to make some modifications at the infra level to manage the archiving, 351 00:39:36,240 --> 00:39:48,880 in particular the limits at the PHP level, etc. And the limits at the level of memory and 352 00:39:48,880 --> 00:39:56,840 the max time. We had cases where the generation took more than an hour, 353 00:39:56,840 --> 00:40:00,200 so we had to optimize that a little bit. And that's one of the ways to improve 354 00:40:00,200 --> 00:40:05,320 that we currently have. I think we really have to try to export 355 00:40:05,320 --> 00:40:13,240 the archiving task on something else. I mean, try to make the archiving task 356 00:40:13,240 --> 00:40:15,280 really easy, because it's something that is very consumerist. 357 00:40:15,280 --> 00:40:21,160 Do you have any examples of cost-per-year ratios depending on the number of visits? 358 00:40:21,160 --> 00:40:25,400 No, I don't have that kind of information, I'm sorry. 359 00:40:25,400 --> 00:40:33,520 But for an infra that we currently have, for the numbers that I showed you there, 360 00:40:33,520 --> 00:40:46,640 we're on five machines, I think. And all of them, let's say, with physical limits, 361 00:40:46,640 --> 00:40:53,000 so flexible CPU RAM. So when we need to increase the load, either we add 362 00:40:53,000 --> 00:40:59,360 a node when it's necessary, or we see that by increasing only the CPU RAM part, 363 00:40:59,360 --> 00:41:03,680 we can absorb the load. Do you have a data system that warns you 364 00:41:03,680 --> 00:41:07,560 when an analyst meets an error page during the report display? 365 00:41:07,560 --> 00:41:14,840 Indeed, in this case, we don't want to be alone. So we actually have monitoring 366 00:41:14,840 --> 00:41:22,360 on the pages. We use classic tools, like Uptime Robot, 367 00:41:22,360 --> 00:41:30,200 PayGearDuty, things like that, that allow us to have errors. 368 00:41:30,200 --> 00:41:36,320 It detects the errors when we make requests. But we also have a system of... 369 00:41:36,320 --> 00:41:42,960 We work, we are very close to our clients, so we work with our clients 370 00:41:42,960 --> 00:41:50,280 with chats, etc. So as soon as they have a problem, we are directly 371 00:41:50,280 --> 00:41:54,560 guided by them to try to solve the problem. Knowing that each time, 372 00:41:54,560 --> 00:42:00,080 there are small positive faults. The last time, it was the famous real-time 373 00:42:00,080 --> 00:42:06,720 counter that was blocked. The fear was that the tracking queue 374 00:42:06,720 --> 00:42:10,800 didn't work anymore, but the data was in the queue. It's just that 375 00:42:10,800 --> 00:42:18,280 Matomo couldn't manage all the data in the database. 376 00:42:18,280 --> 00:42:20,480 Okay. Valentin, can you hear me? 377 00:42:20,480 --> 00:42:21,480 I can hear you very well. 378 00:42:21,480 --> 00:42:29,480 Okay. I'm focusing on this question because sometimes, as a Matomo user, 379 00:42:29,480 --> 00:42:33,480 we often see a message in red that appears in the reports saying that 380 00:42:33,480 --> 00:42:37,480 the server is not powerful enough or that there is an archiving problem. 381 00:42:37,480 --> 00:42:43,080 What I noticed in my clients is that when they encountered this problem, 382 00:42:43,080 --> 00:42:46,680 they weren't questioning Matomo. They were just saying to themselves, 383 00:42:46,680 --> 00:42:50,680 okay, I have to do this request later. They didn't dare to directly 384 00:42:50,680 --> 00:42:54,680 warn the company that manages Matomo. I wanted to know if, 385 00:42:54,680 --> 00:42:58,680 in the monitoring that you talked about a few seconds ago, is it 386 00:42:58,680 --> 00:43:04,680 this type of error message in red that would appear and ping you 387 00:43:04,680 --> 00:43:10,680 and tell you that the person has encountered an error and that it motivates you 388 00:43:10,680 --> 00:43:14,680 to dig and find out how it is possible that they have encountered an error message. 389 00:43:14,680 --> 00:43:16,680 Is it this type of error that has arisen? 390 00:43:16,680 --> 00:43:20,680 No, we don't have too much of this type of error that has arisen, 391 00:43:20,680 --> 00:43:28,680 knowing that we regularly check the application logs, 392 00:43:28,680 --> 00:43:31,680 we check the logs of the machine to make sure that we don't have 393 00:43:31,680 --> 00:43:35,680 abnormal errors on the classic execution of the application. 394 00:43:35,680 --> 00:43:42,680 I admit that, for the moment, this kind of problem hasn't arisen too much. 395 00:43:42,680 --> 00:43:46,680 There is a client who has raised this type of problem on a small infra, 396 00:43:46,680 --> 00:43:50,680 but on a particular case, that is to say that the URLs are extremely long. 397 00:43:50,680 --> 00:43:57,680 It seems to me that the current limit in Matomo is 1024 characters or 4087, 398 00:43:57,680 --> 00:44:02,680 but the fact that we have a lot of URLs that are very long, 399 00:44:02,680 --> 00:44:09,680 it slows down the archiving a lot and it creates a problem in the interface. 400 00:44:09,680 --> 00:44:16,680 So, here we have customer feedback and that's how we realized that, 401 00:44:16,680 --> 00:44:21,680 for even small infra, sometimes it is necessary to drastically increase the resources 402 00:44:21,680 --> 00:44:25,680 at the PHP level to be able to execute the archiving normally. 403 00:44:25,680 --> 00:44:31,680 Okay, it is currently 9.45, we still have maybe 5 minutes to take a few more questions. 404 00:44:31,680 --> 00:44:34,680 We have a new one that has just arrived in the chat. 405 00:44:34,680 --> 00:44:42,680 How do you manage the archiving, do you cut it by segment or year or do you execute it normally? 406 00:44:42,680 --> 00:44:48,680 So, we actually execute it normally, we don't have too many problems with that. 407 00:44:48,680 --> 00:44:55,680 The screen that I sent you, the image that I sent you with our results 408 00:44:55,680 --> 00:45:01,680 of more than 130 million actions, etc., it dates from October. 409 00:45:01,680 --> 00:45:06,680 And so, we are going to work with the teams, our customers, 410 00:45:06,680 --> 00:45:14,680 to discuss the improvement of archiving on this side. 411 00:45:14,680 --> 00:45:17,680 It's going to be a discussion with them. 412 00:45:17,680 --> 00:45:22,680 But otherwise, we normally execute the archiving. 413 00:45:22,680 --> 00:45:24,680 That's it. 414 00:45:24,680 --> 00:45:33,680 Great, so I'm looking to see if we still have a few more small questions coming in. 415 00:45:39,680 --> 00:45:47,680 So, I see that Alexandra, we are told that it is still in writing, in the chat. 416 00:45:47,680 --> 00:45:56,680 So, I take this opportunity to announce the two conferences that will follow right after, 417 00:45:56,680 --> 00:45:58,680 on the costs of Deezer. 418 00:45:58,680 --> 00:46:01,680 You have one on data visualization. 419 00:46:01,680 --> 00:46:04,680 It's a conference in English, but it's done in French. 420 00:46:04,680 --> 00:46:09,680 So, if some of you are not very comfortable with English, 421 00:46:09,680 --> 00:46:14,680 I don't think it's a problem to follow the conference that's coming. 422 00:46:14,680 --> 00:46:18,680 Otherwise, you have the opportunity to go to a workshop. 423 00:46:18,680 --> 00:46:21,680 In this workshop, the places are limited. 424 00:46:21,680 --> 00:46:24,680 That is to say that it is really the first to arrive, the first to serve. 425 00:46:24,680 --> 00:46:31,680 And the workshop, as a reminder, we are based on 25 people who connect. 426 00:46:31,680 --> 00:46:37,680 The workshop is on Implementing Advanced Privacy, Preserving Analytics on Mobile Desktop and Server. 427 00:46:37,680 --> 00:46:43,680 So, probably more a conference that is related to private life. 428 00:46:43,680 --> 00:46:47,680 And so, it will be a feedback on that. 429 00:46:47,680 --> 00:46:49,680 So, it will be by Nathan Freitas. 430 00:46:49,680 --> 00:46:54,680 And so, in room number one, the data visualization part will be with, 431 00:46:54,680 --> 00:46:58,680 precisely, the presentation of a free software called MetaBase, 432 00:46:58,680 --> 00:47:01,680 that maybe some of you know. 433 00:47:01,680 --> 00:47:06,680 And it is presented by Baptiste Stephen, who works at Open Source Politics. 434 00:47:06,680 --> 00:47:10,680 I look one last time if there are any questions in the chat. 435 00:47:10,680 --> 00:47:15,680 Well, yes, we have another question from Alexandra. 436 00:47:15,680 --> 00:47:19,680 Have you encountered errors like MySQL Server has gone away, 437 00:47:19,680 --> 00:47:22,680 so there is an error that often comes up, 438 00:47:22,680 --> 00:47:26,680 during archiving after having invalidated periods? 439 00:47:26,680 --> 00:47:29,680 No. 440 00:47:29,680 --> 00:47:32,680 So, it happened to us, but we increased, 441 00:47:32,680 --> 00:47:37,680 we increased in particular the number of connections at the MySQL server level. 442 00:47:37,680 --> 00:47:42,680 And I do not think we encountered this problem too much. 443 00:47:42,680 --> 00:47:46,680 It seems to me that we were a little spared on this thing. 444 00:47:46,680 --> 00:47:51,680 We had to meet it quickly, but it is not something that shocked me 445 00:47:51,680 --> 00:47:55,680 in the maintenance and management of the infrastructure at the moment. 446 00:47:55,680 --> 00:47:57,680 Sorry. 447 00:47:57,680 --> 00:48:01,680 Great. But listen, thank you very much, Valentin. 448 00:48:01,680 --> 00:48:06,680 Thank you very much to all of you for choosing this conference. 449 00:48:06,680 --> 00:48:09,680 We now have 12 minutes for a coffee break. 450 00:48:09,680 --> 00:48:14,680 Do not hesitate to interact in the coffee room if you want to discuss with other participants 451 00:48:14,680 --> 00:48:16,680 or go for a real coffee. 452 00:48:16,680 --> 00:48:22,680 Or, and do not forget, you can also go to meet.matomocamp.org 453 00:48:22,680 --> 00:48:27,680 to also interact directly via webcam with other participants. 454 00:48:27,680 --> 00:48:32,680 I give you all an appointment at 10 o'clock for the next conference game. 455 00:48:32,680 --> 00:48:37,680 Thank you all.