SAMUEL DEGUELDRE: 0:00

Most performance problems happen on And so if the client is slow, it's because the server is responding slowly. And at that point you don't have a JavaScript performance problem. The life cycle on the Python side is quite short, but on the JavaScript side, people will often come first thing in the day open up Odoo, and they leave it open all day. And so we have to be more conscious of memory than we we have to be on the Python side. There are hard leaks, but hard leaks are runtime bugs. It's a bug in Chrome itself, and it can happen. But it's I don't think I've ever found one. Basically, all the things that you should see in the memory snapshot are Chrome internals like compiled code, interned strings, that sort of stuff. This is nothing to worry about, but if you see objects or components or Dom elements, this is when you know you've got a memory leak. It's not black magic like most things in software. You just have to be interested, do some research, and then just do it a lot. You do it a lot and then you get good. Uh, most of the tools are pretty intuitive, like especially all of the Chrome dev tools. They're not super well documented, but most of the things are kind of self-explanatory. It's just that you have to to, like, play around in the UI to find all of the options. But once you found them, uh, you're pretty much golden.

OLIVIER COLSON: 1:13

Hi, everyone, and welcome to this new Today I have the pleasure to discuss an always trending topic JavaScript, with my guest, Samuel Degueldre, developer at Odoo. We'll go through various subjects, including Owl, performance optimization, and many more, so make sure to stay until the end because you might learn a lot of things. Ready? Let's go. Hi, Sam.

SAMUEL DEGUELDRE: 1:36

Hi, Olivier.

OLIVIER COLSON: 1:36

So it's not your first time with with all that. You were in the first episode of Odoo unplugged. But it's the first in the podcast, right?

SAMUEL DEGUELDRE: 1:44

Yeah.

OLIVIER COLSON: 1:45

Welcome.

SAMUEL DEGUELDRE: 1:46

Thank you.

OLIVIER COLSON: 1:47

So, what are we going to talk about

SAMUEL DEGUELDRE: 1:50

Um, well, I'm told I've been invited so probably that.

OLIVIER COLSON: 1:56

Yeah, I don't have any surprise. We don't change the subject like that. Come on. Uh, yeah. I cannot do that. Uh, maybe first introduce yourself a little bit for people not remembering you from, uh, unplugged.

SAMUEL DEGUELDRE: 2:06

Yeah. And also people who didn't watch

OLIVIER COLSON: 2:10

Come on, everybody watched it.

SAMUEL DEGUELDRE: 2:12

I mean, I'd like to think that I'm a the case just yet.

OLIVIER COLSON: 2:17

Not yet. But after this episode, you

SAMUEL DEGUELDRE: 2:21

So I'm Sam, I've been working at Odoo part of the JavaScript framework team. So the team that, uh, writes and maintains all of the, the framework that we have for the client side, um, and I'm one of the people on this team that specializes in performance. And so whenever we find performance issues in the framework or in JavaScript in general, um, I'm kind of the go-to person to investigate t hese issues or help people when they're making new developments and the performance just isn't quite up to snuff. Um, I can come in and help them. And so, yeah.

OLIVIER COLSON: 3:01

How did you end up being that person? You have a specific background or not really?

SAMUEL DEGUELDRE: 3:06

So kind of, uh, just being interested around with the tools that exist. Um, at some point, I developed the skill set, and it's kind of like the ball gets rolling, right? Once, you know, a little bit of this stuff, people can come to you with their issues. And so you learn more and then, uh, now I'm here.

OLIVIER COLSON: 3:26

Um, and how often does it happen that to solve in really the JavaScript side? Uh, because we often talk about the performance of the server, of the SQL queries, whatever the volume of data. But how often does it happen in the browser itself?

SAMUEL DEGUELDRE: 3:41

So first of all, you always have to be You can't just write code. And then when it's really, really slow, uh, just throw up your arms in the air and say, well, I never could have seen this coming. So, uh, like, being performance conscious is something that happens all the time, but, uh, like actual performance problems that happen in production that make the application difficult to use or maybe even unusable. They're quite rare. Um, and then when we find them, uh, well, when we find the issues, uh, it identifying the root cause and then fixing it is generally not that difficult. So it's not something that's ever-present. Uh, although I have to say, um, since Odoo has been growing a lot and we, we have been recruiting more and more developers, and also as we've increased the functional coverage in like the, the width of the functional coverage has kind of, uh, stopped growing outwards. And so now we're going in depth. And when you want to go in-depth, uh, one of the big things is user experience. And to have a great user experience you tend to write more JavaScript. And so we're finding more issues with JavaScript code just because we're writing more JavaScript recently.

OLIVIER COLSON: 4:52

Also just for, you know, more Yesterday I wrote an old widget for a field for something coming in, in accounting, and yeah, I had to write it. And a few years ago, it was never like that because we never focused on more detailed stuff like that. And now we've reached a maturity and the size of the company where it's possible to allocate resources for that. And so indeed, uh, maybe I did complete shit and I will need your help at some point. Who knows?

SAMUEL DEGUELDRE: 5:24

So hopefully, we've written the to worry too much about performance. Generally, like the main extension points are places where it's kind of hard to mess it up right when you're registering a field. The big performance concern around, uh, fields specifically is data fetching, and hopefully, data fetching should be all handled by the framework, which means that you don't have issues where you have a request. And then after that request comes back, you fire off another request where you have sequential data dependency that causes stuff to load for a while because you have to talk to the server back and forth many, many times. Uh, generally, this is just handled by the framework. And so you don't really have to worry about it.

OLIVIER COLSON: 6:08

So often the amount of data that you be small enough that you don't have big issues performance-wise. If I get it, unless you do tons of queries.

SAMUEL DEGUELDRE: 6:20

Yeah. So and this is something that reasons where most people don't have to worry about JavaScript performance too much is that generally, uh, in on the server, if you need to run an aggregate query and, I don't know, find the the average price of, uh, a transaction over, I don't know, two years. You have lots and lots of data to come through. And so it takes a lot of time. But on the client side we have two advantages. One is, well most of the work should be done on the server anyway. And so if there's a performance problem, it probably happened before the data ever reached the client. And the other is that you can only ever display so much data on one screen. And so. Even like one of the easiest strategies is to just not fetch too much data. And this is typically what we do. For example, in the list view, we only show the first 80 records because it's completely useless to load a thousand records if the user is never going to be looking at them. And so we just have a page, and you can go to the next page and show like 80 records at a time. And it's still customizable. And so we still have to worry that the performance is good even when you load a lot more records. But the default case doesn't really need to worry too much.

OLIVIER COLSON: 7:35

And you, anyway, don't load them all sense. It would be unusable. You will never display 10,000 records in one view, and expect the user to find that user-friendly. It's.

SAMUEL DEGUELDRE: 7:45

Yeah, exactly. At that point, like you experience anyway. So this is why you have pages, and this is also why you have searching that sort of stuff. We don't expect users to just scroll their records until they find what they want. They use the search. And so mostly, uh, the fact that we even display many records at all is the latest records are generally useful. And then, uh, sometimes you, you, uh, write a search query, and you expect multiple records to be returned and then, but you don't expect like, hundreds of records generally.

OLIVIER COLSON: 8:18

Okay. What would be the typical So there is something working badly in some JavaScript code. What is it typically?

SAMUEL DEGUELDRE: 8:28

So. The typical like performance Things that run too slow are not very different. Uh, on the client side, as on the server side, obviously we don't have to worry about querying the database and stuff, but sometimes, uh, like the most performance problems happen on the server. And so if the client is slow, it's because the server is responding slowly. And at that point, you don't have JavaScript performance problems. But when you do have problems, they're generally the same kind of dumb mistakes. Like most of the issues that we find when someone, uh, reports that, uh, the application is behaving, uh, slowly, uh, or things take a while to display that sort of stuff. It's, it's just a dumb loop in a loop where you have some behavior that's n squared. And sometimes, it's a little less obvious because you have a loop. And inside of this loop, you call an array method that iterates over the array. But basically, it's the same issue. So most issues are just really dumb. I don't think I've ever found a performance problem. And I thought, wow, that's just really so subtle.

OLIVIER COLSON: 9:35

Okay. And how do you debug them then? Uh, is there any specific thing to JavaScript, or is it like for any other language.

SAMUEL DEGUELDRE: 9:41

Uh, yeah. So for, for performance, uh, like execution time, uh, like most languages, uh, you would use a profiler, and the Chrome DevTools have a great built-in profiler. So if you know how to use a profiler, uh, you'll feel right at home. It has some nice features. You just press record, and do some stuff. Uh, stop. And then you get a nice, uh, flame chart or flame graph. I never know which one of the two it is. So one graph, I think. Yeah. So there's, there's two. And I think the one that Chrome has is a flame graph. And speed scope displays a flame chart. So you don't agree. The, the subtle difference is that a flame a flame graph is in time order. So you have each successive function call at the point where it was called, and you see how long it takes. And then, in the flame chart, you collapse stack frames that are the same function. And so you can see overall the execution time, which function took the most time.

OLIVIER COLSON: 10:39

Okay, okay. And you kind of need both you like the sequence of what you're doing and the other will will tell you you have to watch this function uh, in particular. Okay. Uh, and so you were mentioning runtime. You also have to manage the resources around your code. So like memory. Yeah. Uh, is it a big problem with JavaScript, then?

SAMUEL DEGUELDRE: 10:59

So I think this one is a bigger the server side because, like on the Python side, we have we spawn workers, and then uh, we regularly respawn them after a while. Um, and so when a request comes in, um, generally, all of the memory that's going to be allocated is going to be tied to that request. And as soon as the request has been responded to and it goes out of scope, uh, all of the associated memory just vanishes. It's garbage collected, and it works quite well. So the life cycle on the Python side is quite short. Uh, but on the JavaScript side, like, people will often, uh, first thing in, in the day, open up Odoo and they leave it open all day. And so like, we have to be more conscious of memory than we, we have to, to be on the Python side. Um, so it's more often a problem. Yeah, um, like just long-lived applications in general, um, or long-lived execution contexts. Right.

OLIVIER COLSON: 12:03

And so what can what can happen really can stay in the memory. Do you have do you have memory leaks in JavaScript like you would have in C? I don't know, uh.

SAMUEL DEGUELDRE: 12:12

So it depends what you call a memory Uh, so there's.

OLIVIER COLSON: 12:15

Sort of have pointers to three here. So I guess it's easier, but still. So you can...

SAMUEL DEGUELDRE: 12:19

It can happen, right. There are hard bugs. It's a bug in Chrome itself. And it can happen. But it's I don't think I've ever found one. Um, and then they had you have what we call memory leaks when we discuss internally soft leaks where you have objects that are still reachable. So if you were to try to access them from JavaScript, there is a, there is a path from a global object. So like the window or the the document object or some HTML element that's in the Dom, these are all reachable. And so any object that's reachable from these objects is also considered reachable and cannot be garbage collected. And a lot of the time the issues that we found are some objects are still reachable but are actually no longer useful. And so this is the biggest source of memory leaks that we have. And uh, typical examples can be you have an event listener that hasn't been cleaned up correctly. And so this is both a memory leak but also a problem of correctness. Right. Because you have an event listener that you don't want to get called, but it might still get called because it's he's an event come through. So this one is event listener on the. HTML elements, but what's more common is, for example, an event listener on the window. So components, sometimes want to get mounted and listen for events on the window. For example, they want to know if the window is getting resized. And so they'll attach an event listener on the window. And if ondestroy they forget to clean it up, it means that since the handler is allocated, the component is still allocated because it's reachable. Like if an event happens, it's going to call code inside of the component. And then, if the component is reachable, the application is reachable. If the application is reachable, all of the components from that point in time are still reachable. And also all of the associated Dom nodes. So it's it can be a lot of memory for just forgetting to clean up one event listener. And so that's why we also in the framework have utility functions that will it can be really tricky to to find.

OLIVIER COLSON: 14:19

I mean you can have one path like well stay into the memory. So how do you find them?

SAMUEL DEGUELDRE: 14:29

So on the one hand you might think, leaks all of this memory. So it's going to be difficult to find. But generally what you do is in a Chrome DevTools, you take a memory snapshot and then you can find this huge amount of leaked memory where you know, these components, they're destroyed, so they shouldn't be allocated. And then you see how they're reachable. And by seeing how they're reachable, like most of these, these links are obviously it's like a component has a parent. So it's reachable through its parent. And then you just go down the chain until you find something where it shouldn't be there. And the memory snapshots are generally, um, sorted by the depth from what they call garbage collector routes, GC routes. And so...

OLIVIER COLSON: 15:14

So the number of links to reach them

SAMUEL DEGUELDRE: 15:17

Yeah, and so since they're sorted by, Yeah, increasing length rather than increasing depth, it means that the objects that are leaked that are responsible are likely to be at the top. And so you can find them like this.

OLIVIER COLSON: 15:34

And so you compare multiple snapshots them and what stays. Something like that?

SAMUEL DEGUELDRE: 15:38

Yeah. So the Chrome dev tools have take a snapshot at one point in time, then take a different snapshot, and then you can see which objects existed in one snapshot and not in the other. And so what you would typically do is you have a flow that, you know, leaks some memory. Because if you do it repeatedly, uh, the, the memory usage of the application balloons. And so what you do is you first do the flow once, just so that all of the things that need to be allocated one off for the duration of the application are, are allocated. And then you take a snapshot and then you do the flow again. Then you do a second snapshot, and then you do the flow a third time. And you take a third snapshot. And now you, you see the, the objects that are still alive that were allocated between the first and the second and all of these objects, because you've done the flow a third time, they should have been garbage collected and reallocated, but they wouldn't be the same objects. Right? So between snapshot one and snapshot two, there should be basically nothing. Uh, basically all, all the things that you should see in the memory snapshot are Chrome internals like compiled code, interned strings, that sort of stuff. This is nothing to worry about, but if you see objects or components or Dom elements, this is when you know you've got a memory leak. Clean things properly. Yeah. And at that point, you can start, uh, looking at the chains of retainers and finding finding out what's retaining the memory.

OLIVIER COLSON: 17:03

Okay. I would say that that's like the you can use, maybe before that, because then you need to know what flow is causing the problem? So you need to have already an idea of where, where the issue is, is, uh, is there something else that you can use to track? Well, what part of your, of your website, uh, is, is causing issues?

SAMUEL DEGUELDRE: 17:27

Yeah. So as I was saying, the Chrome uh, as their name might suggest. Um, and specifically for memory, there are two other, uh, types of, uh, measurements that you can make. Uh, one of them is, um, allocation instrumentation on timeline. And this one, uh, is a great tool to identify, uh, like the, the point in time where memory is actually leaked. So when you start recording, you get this sort of time series where every time memory is allocated, Chrome will display, uh, a blue bar, vertical bar, um, that corresponds to the amount of memory that was allocated. And as the application keeps running, this bar will become partly gray colored to represent all of the memory that was allocated that has now been reclaimed. And so when you see a huge blue bar, and then it becomes entirely gray, that just means you're allocating a lot of memory, and then you're allocating it correctly, and there's no problem. But if there's a remaining blue bit, then that means that this memory is. And so you can select this in the timeline and identify these objects. And it's going to look very similar to the heap snapshots.

OLIVIER COLSON: 18:38

But you don't have to bother doing The snapshots and comparing them. And so it's like it gives you a faster view of where there could be a problem. And then you can investigate, I guess, with the snapshots, right?

SAMUEL DEGUELDRE: 18:49

Yeah. So it's I would say it's, it's complete because obviously you haven't done the entire flow. And sometimes objects can remain in memory and linger for a while, but not forever. Like, for example, it could remain for half the flow or three-quarters of the flow, but in the end, it's going to be garbage collected. There are also sometimes things that are kept in cache for, uh, some amount of time. And so, um, yeah, I would say it's uh, it's a more blunt tool, but it's also much faster to find issues if there's something obvious.

OLIVIER COLSON: 19:22

Okay, okay. Is there something else we

SAMUEL DEGUELDRE: 19:26

Yeah. So the, the third one, uh, like is, I would say, much less useful. Um, it's something that allows you to record, um, allocations, but it doesn't keep the objects in memory. It doesn't really do a snapshot. And so it's, it's more about seeing which functions allocate a lot of memory. And this one is less useful for, uh, finding memory leaks. Uh, but it can be useful for detecting things like, uh, what's called garbage collector thrashing. Uh, that's what happens when you're allocating a lot of memory and then garbage-collecting it immediately. And because the garbage collector has to, at some point, pause the execution of the JavaScript to do the actual garbage collection. Um, it can have a big performance impact. But this one is kind of I would say generally it's pretty obvious even without using this tool, where where your code is allocating a lot of temporary objects. But if you're not finding them, then you can use this one. And also, um, if you want to automate stuff, uh, this one has less impact on performance. And so, for example, we have tools that we use for testing that just simulate a bunch of user interaction for flows. It's possible to instrument, uh, a chrome headless to, uh, record this kind of, uh, of profile. And then you can have data about much longer flows than would be practical to do by hand. So it can be useful. I think there are kind of, uh, from most useful to least useful in the Chrome UI. So heap snapshot is most useful. Uh, but sometimes it can take a little bit to set up. Uh, allocation instrumentation on timeline is, uh, blunt but fast. And then the last one is kind of just it's more about performance problems that are caused by memory allocations than actual memory issues. Right.

OLIVIER COLSON: 21:14

Okay. Outside of the dev tools, are use or install towell, track performance issues from a memory perspective?

SAMUEL DEGUELDRE: 21:24

Yeah. So, um, a while back, I was, uh, memory leaks, uh, involving weakmaps. And what I started doing was I was writing some JavaScript code to parse the heap snapshots that are emitted by Chrome. Like, you can export them. For example, if you want a flame chart in speed scope, you can export memory snapshots, uh, or performance snapshots. Sorry. Um, and yeah, I guess also memory snapshots. But you can export performance, uh, snap snapshots and memory snapshots, and performance snapshots. You can load in speed, scope and get something familiar. Because I think that's the tool that we use for displaying Python profiles or the one that we recommend. So if you want to keep using the same tool, you can and it can help you dig down. And for memory, there's this tool written by Facebook, which is an npm package called memlab. And it has a lot of tools to interact with memory snapshots because the memory snapshots that are emitted by Chrome, they're JSON, but they are also like a specific kind of formatted JSON. Um, and it represents a graph in the least usable way. So you have to spend a lot of effort just trying to reconstruct the graph in a way that's it's easy to query and memlab. They've kind of done that for you already.

OLIVIER COLSON: 22:39

So they give you an API to interact do whatever analysis you want on them to find what the problem is.

SAMUEL DEGUELDRE: 22:47

Yeah. And also for the like the what I differential snapshots where you take a snapshot, take another snapshot, take a third snapshot and then compare the first two, um, they, they have a programmatic API where you can, um, do some flows. Uh, some I think it's based on, on selenium. So it's very similar to our tour system, where you tell it to interact with a page in a certain way and take snapshots at various points. And this can be used, for example, to, to automate, uh, finding, finding memory issues and or also to obtain metrics. For example, if you wanted to see, uh, how memory usage, uh, evolves across a certain flow. So it's very useful for automation and it's very useful for advanced analysis. But yeah, I would say, uh, by the point you're using memlab, you're probably already you probably already know what you're doing.

OLIVIER COLSON: 23:38

Okay. That's that's a good marker. Uh, all right. So we're reaching the end of the episode, maybe before ending, could you give us like, one anecdote, something that happened inside of Odoo where you had a big performance issue like that in JavaScript. So be it on runtime or memory? You decide. Uh, and what happened with it?

SAMUEL DEGUELDRE: 23:57

Yeah. Um, so at some point, I was um, their JavaScript code base to update to the latest version of Owl. And, um, for some reason, because I was on the PoS team, uh, there was some performance issue reported by a customer. And Antony, since I was on the post team at the time, he said, well, uh, take a look at it. And so, uh, I took a look, look at it. And so the PoS is kind of one of those few places in Odoo where because we wanted to work offline, uh, we need to load all of the data so you, you get the same kind of performance problems that you get on the server just because you have a lot of data. Like when I was testing, I was testing to, to recall, uh, pos point of sale, the point of sale. Sorry. Thank you. And so you need to work offline. Why? Um, because in some places, uh, the internet connection is spotty. So, for example, people, uh, they work with their tablets and, uh, like, they have no Wi-Fi, or they're using 4G.

OLIVIER COLSON: 24:56

And they could lose the connection. And so you need to need to, to keep some kind of cache, uh, of what they're doing so that you can resynchronize with the server afterwards.

SAMUEL DEGUELDRE: 25:03

Yeah. And so we load all of the in the browser, uh, in the background, and the, the issue that was reported is that as things were loading in the background, the application became more and more unresponsive. And I think we were in on the order of 100,000 or 200,000 customers and products. And so I just did a simple profile, and it turned out that, uh, in order to, to have performance searching for partners and products, uh, we constructed a large string that represented all of the partners and the products, with a few, like sigil inside the string to be able to find which customer or which product was represented by this section of the string. And the reason we did that is that, uh, you can use regexp to search inside this huge string very, very fast because regexes are they're compiled by the regex engine of the browser. And so they're really, really fast. And the issue is that, well, we have this, all of this information about uh customer, for example, it can be something like 100kB with uh, their name addresses, whatever. Um, and we're every time that we're loading a new batch of partners in the background, we're building this string for this partner and appending it to the existing string. And so if you know anything about, like, vectors or dynamic arrays and, in this case, strings, it's kind of the same. We're just adding stuff at the end. And to do that you have to deallocate the memory and allocate new memory where this new data will fit. And we have 100,000 partners. So the string is hundreds of Meg... Well, not hundreds. It was around ten megabytes for one string.

OLIVIER COLSON: 26:49

A string taking megabytes is already

SAMUEL DEGUELDRE: 26:51

Yeah, yeah. And so it's just, uh, as I was saying, it's a pretty dumb issue, right? If you know that you have an object that needs to grow a lot, then instead of using something like a string that you have to reallocate every time, it would make more sense to have a data structure that's actually appropriate to grow progressively from the end. And so, what I did is I simply split up the the string into chunks. So instead of having one huge string for 100,000 partners, for every block of 1000 partners ID, we have one string. And so that means that we can still get most of the performance benefit from regex, where we just, uh, since...

OLIVIER COLSON: 27:27

You still need to iterate on those, But since it's really fast, you don't care.

SAMUEL DEGUELDRE: 27:31

Yeah. So if you have 100,000 partners, thousand partners, you have to loop 100 times so you don't lose much time in this loop. And you still get the benefit of regex, uh, for, for the search itself.

OLIVIER COLSON: 27:44

Okay. That's an interesting one. You said it was never like niche things and then pretty complex ones. But this one, it's... I mean, it's exotic.

SAMUEL DEGUELDRE: 27:52

Yeah. Yeah, it's one of the most Most of the issues that we find are much, much dumber.

OLIVIER COLSON: 27:58

Okay. Um, as a conclusion now, um, someone, like wanting to become like the, the, the new JS perf expert? Uh, how can it do that? Uh. Just trying?

SAMUEL DEGUELDRE: 28:11

Yeah, honestly, it's it's not black Uh, like most things in software, you just have to be interested, uh, do some research and then just do it a lot. You do it a lot, and then you get good. Uh, most of the tools are pretty intuitive. Like, especially all of the Chrome dev tools. They're not super well documented, but most of the things are, kind of self-explanatory. It's just that you have to like play around in the UI to find all of the options. But once you found them, you're pretty much golden.

OLIVIER COLSON: 28:38

Mhm mhm. And anyway you've given like that people can just have a look and investigate themselves. And I'm pretty sure with that, uh, they can take the first steps to becoming experts in the field. Thank you very much for your answers, and see you in the next episode, hopefully.

SAMUEL DEGUELDRE: 28:55

Thank you for having me.

OLIVIER COLSON: 28:56

And that's a wrap for this episode. If you learned things and would like to go deeper into the JavaScript topic, we already published several episodes on this matter, so I suggest you go listen to them. The links are in the description. On my side, I thank you for staying until the end. Have a great day, and till next time. Cheers!