Code and the Coding Coders who Code it
We talk about Ruby, Rails, JavaScript, and everything in between. From tiny tips to bigger challenges we take on 3 questions a show; What are you working on? What's blocking you? What's something cool you want to share?
Code and the Coding Coders who Code it
Episode 45 - Stephen Margheim
Stephen Margheim, a celebrated figure in the Ruby and Rails community, returns to unravel the fascinating intricacies of his latest project—writing a parser for SQLite's SQL dialect in Ruby. He shares his enlightening journey of translating complex SQL syntax, which at first seemed a simple endeavor but soon unfolded into a realm of deep learning and unexpected challenges. Alongside this, Stephen collaborates with Aaron Francis on "High Leverage Rails," a video course designed to spotlight the synergy between Rails and SQLite, offering a treasure trove of insights into developing high-quality applications.
We dive into the nuanced world of SQL parsing, where Stephen candidly recounts the arduous process of porting SQLite's lexer and parser into Ruby. What began as a straightforward task quickly turned into a labyrinth of complex syntax and discrepancies that required astute attention and incremental progress. He reflects on the absence of a fully compatible SQLite parser in any language, emphasizing the significance of open parsers like Postgres in creating a robust ecosystem for tools and libraries.
Stephen's excitement is palpable as he discusses Quickdraw, a groundbreaking testing framework that revolutionizes testing in multi-core environments. This innovation, along with the anticipation for RailsConf 2025 in Philadelphia, paints a bright future for the Rails community. With rich discussions on parsing, testing, and upcoming Rails events, this episode promises to inspire and engage both seasoned developers and newcomers to the Ruby and Rails landscape. Join us for an episode filled with excitement, insight, and a glimpse into the future of Rails development.
Honeybadger
Honeybadger is an application health monitoring tool built by developers for developers.
Honeybadger is an application health monitoring tool built by developers for developers.
Disclaimer: This post contains affiliate links. If you make a purchase, I may receive a commission at no extra cost to you.
Ready to start your own podcast?
This show is hosted on Buzzsprout and it's awesome, not to mention a Ruby on Rails application. Let Buzzsprout know we sent you and you'll get a $20 Amazon gift card if you sign up for a paid plan, and it helps support our show.
Hello everyone and welcome back to Code and the Coding Coders who Code it. I'm your host, drew Bragg, and I'm joined today by repeat coding coder, stephen Marghime. Stephen was on in episode 33, but for anyone who has not heard that episode yet, I do recommend you go listen to it. But, stephen, would you please introduce yourself to anyone who might be new to you.
Speaker 2:Well, first and foremost, I am a coding coder, at least until AI codes all the code. But primarily, my public persona at least, is most associated with working on SQLite in the Ruby and Rails community. So I did a lot of work in the SQLite features for Rails 8. I have a number of gems as well in that ecosystem and that's what we talked about in the SQLite features for Rails 8. I have a number of gems as well in that ecosystem and that's what we talked about in the previous episode.
Speaker 1:Yeah, we did a lot of SQLite in the last episode and all the work that you've done to that and you've been busy since then. So I know you know how this is going to go. But for anyone who might be new to the show, I'm going to ask Stephen three questions. I'm going to ask him what he's working on a bit of a loaded question what kind of blockers he has. If he doesn't have a current blocker, he can talk about a recent blocker he had, how he went about solving it. And then my favorite question is what's something cool, new or interesting that you've recently learned, discovered? Built Doesn't have to be coding related, but this is Code Encoders, so it totally can be. So I know we have a lot to dive into. So, stephen, what are you working on?
Speaker 2:Yeah, it's turned into a busy start of the year. I have stumbled somewhat ass backwards into writing a parser for SQLite's dialect of SQL in Ruby. That has also meant that I am learning how parsers work and how to write one. That has been a lot of my open source work. And then just this week I finally got to start talking about a video course that I recorded in November in Dallas with Aaron Francis, the internet's favorite dad, who was on an episode 39, also Love Aaron.
Speaker 2:Yeah, you have all the big celebrities, I try. Yeah, that was a ton of fun. We connected online and I was able to fly down to Dallas after RubyConf where you got to keynote. We were hanging out, and so that next week took the plane down to Dallas and recorded in his studio a video course on building high-quality Rails applications with SQLite, and so that gets to be released next month. And they did the big hype launch trailer released earlier this week. That was exciting to see.
Speaker 1:Yeah, I'm looking forward to that. I think Aaron's always put out really quality content and he really seems to know what he's doing and I think the approach he's taking to kind of empowering and enabling others to put out that same level of high quality content is really awesome. Whose idea was it? I mean you said you connected and were able to fly down there. Did he reach out to you? Did you reach out to him? Was it kind of like both of you were like this sounds awesome for his SQLite?
Speaker 2:course. I did an episode with him on his database school podcast and so we got to connect in person, quote unquote at that time and so we kept chatting after that and I honestly don't recall which one of us broached the general subject first, but probably it was him, and I think he just roughly asked like hey, do you have any? And I think he just roughly asked like hey, do you have like any ideas for courses? And I told him that, yeah, I had been thinking about how I wanted to sort of package together a lot of the different tools and experiments and things I've written about on my blog into like a cohesive narrative to really clearly lay out how powerful it can be for an individual or for a small team to just lean on these two tools instead of trying to get reasonably good at Rails and Redis and Sidekick and Active Storage and S3 and all of the different things. To tell this story of how you can gain real leverage.
Speaker 2:That's the title of the course High Leverage Rails. Check it out at highleveragerailscom. Yeah, he got really excited by that idea, so we just started chatting, chatted with his business partner, steve, and everyone was on board, so we found the time where it made sense for me to come over to the US In this case. I was already in the US and put it together.
Speaker 1:This is your first video course, right? Did you go in with sort of an expectation of this is what it's going to be like and it was exactly what you thought it was, or not what you thought it was, or you had no idea what you were getting yourself into like. What was the whole experience like?
Speaker 2:fundamentally I had no idea what I was getting into, but it was useful to have aaron there as a bit of a guide. So the preparation phase was basically just centered around getting a good outline together. So, like, what are the videos that I want to shoot, what's the topic that they're going to cover and what is the sort of narrative arc of all of those videos? So we did a few rounds on that and coming out of that I realized, okay, there's two or three areas where I would benefit from spending a little bit more time exploring in preparation. A lot of it obviously draws on a lot of the work I've already done. So I felt really comfortable in many of the areas that we were covering. But there was a few I did some prep work for, but not too bad.
Speaker 2:That wasn't like months of preparation. We basically had a call and said, like, do we do this in a sprint? Do we do this in the marathon? We decided to do it in a sprint. So we put the whole thing together in a month or less. And then, yeah, when I got to Dallas, the recording process I really had no idea what to expect and it was interesting Basically took one day just to get everything set up, get my machine set up to record and connected into the studio, and then from there I would just lock myself into the studio at eight or nine in the morning and click record, and I recorded like on average 15 videos a day for four straight days.
Speaker 2:It was hectic, but it was interesting as well to know I'm only here for this time. I have these, however many videos. The outline grew a little bit while recording as these things go, but it worked out really well. I don't think it could have been anywhere near as good as it is without Aaron and Steve's help, and together we were able to put it all together pretty quickly, relatively speaking, and I think that that also helped because I wasn't in my own head for too long, stressing it's like hey, we should do this. Yeah, we should do this. Will you be around in a month? I will.
Speaker 1:All right, the stars just aligned for you Makes it easy. When exactly does the course go live?
Speaker 2:I actually don't know the exact date because Steve, the editor, is still finishing up the edits and he hasn't 100% committed, but it is in February. So generally like around-ish Valentine's Day, middle of February, a little bit before, a little bit after.
Speaker 1:Great, awesome Valentine's Day gift to the entire Ruby and Rails community and the SQLite community and everything.
Speaker 2:And it makes a great Valentine's Day gift for any coder who has a significant other. I'm sure they would just love a 60 video three, four hour course on getting into web development.
Speaker 1:Sure, yeah Hell of a lot better than a box of chocolates.
Speaker 2:Exactly so I'd say buy two, three licenses, bring them to your bowling league, pass them out.
Speaker 1:You know how bowling leagues and technology go together.
Speaker 2:Yeah, a match made in heaven.
Speaker 1:So you did this course, which is wild and super excited for it. But you also said that you kind of fell into doing a SQL parser in Ruby and you said it was sort of because of this course. So a little more background needed there.
Speaker 2:Well, the short story is quite short. I full-on nerd sniped myself Nice.
Speaker 1:You're supposed to nerd snipe others.
Speaker 2:I know that's the smart way to do it. That's the smart way to do it. Unfortunately, no one else was stupid enough to be nerd sniped by it, so it was left to me. One of the areas where I prepped for a longer period of time for the course was I wanted to do an initial module really getting into the foundations of how ActiveRecord works with SQLite. So when you write this ActiveRecord command, what SQL query is it generating? How do you read and write? Just sort of tying together those two worlds.
Speaker 2:And as I was digging into that and running some different experiments as these things go, I bumped into some bugs in ActiveRecord. So when I went to fix the bug, I ran a migration to generate a virtual column and my app didn't work anymore. Went and dug in and realized that the way that ActiveRecord introspects to get an understanding of your schema so that it has like all of the Ruby knowledge for your models, like what columns they have and their names and their types and all of the magic that ActiveRecord does for you At the very core of that for SQLite it gets the create table statement string and runs some pretty basic string processing. So it's like what are all of your columns? I know I will find the first open parens. Split the string there and take that leftover string. Split it on commas and take the first quote wrapped words and those will be your column names and it should be fine. And so what I bumped into is sounds fine and that's the thing is. You know it worked. It worked for 20 years until we added virtual column support, where now you can put full expressions as the default expression for a generated column and that expression even could just be a simple function.
Speaker 2:But that function could have two parameters and how do you separate two parameters in a function with a comma? And just broke the string introspection. So I made a PR to fix that with writing some gnarly regexes. But as I was looking at it, it's just like this is just ugly. I know I could write SQL that would break this again and I just felt this pull to like this should be solved properly. What would it take to solve it properly and realizing? Well, it would take, like a parser, like you have to just know how SQLite understands, create table statements, so that like planted the seed for the idea and then it just sort of grew. As these things go, it just grew in my mind. I became a little bit obsessed with it. How hard could it be? That would be useful, that would be interesting.
Speaker 1:How hard could it be? Famous last words.
Speaker 2:Famous last words indeed, and as other coding coders I'm sure will experience, you're on the holiday break. It's so nice to relax. You know what would be great, really relaxing, starting a new project. That would be a lovely Christmas gift to myself. So I decided, like how hard could it be? Let me just like dig into SQLite C code. Let me just see if I can write a lexer. I'll just take its lexer and see if I can port it.
Speaker 2:And I ported it in like a day, I'm like ah, come on, I knew it, this would be so easy. C is not so hard, this isn't so scary. And so I'm like I'm committed, let's do it. And then I open up the parse file. I'm like this is the darkest of dark magic I've ever seen in any of the 400 lines of C I've ever read in my life. So I just immediately ran into a brick wall. But I had that taste. So I decided let's keep slamming my head against this brick wall. And a month later I'm actually on the other side. I am going to finish this parser and I can parse, create table statements now fully and completely, which is exciting.
Speaker 1:The next question I always ask is what kind of blockers do you have? But how did you get through that brick wall? That's a brick wall that I feel like a lot of people would like. Oh, that's a generated parser. Well, f this, I'm out. How did you stay motivated to actually get it done? And like what were the blockers that you ran into and going about solving them?
Speaker 2:So many blockers. I just have to pick a few of the high interest ones. But on the motivation front, I think that it is an important lesson and I have relearned this lesson many times over the years. It really is important to approach problems in incremental steps and give yourself quick wins. I didn't fully know it, but I did have a general sense that lexing is easier than parsing, and so before I even think about the parser, before I even look at the parser, before I've done anything there, let me just go straight into the lexer and I bit off some amount of work that I could chew and it gave me a sense of accomplishment and it gave me a sense of possibility and I was able to build on that foundation. If I had started and my very first step was go and read the parse file in SQLite source, I don't think I would have gotten anywhere because I just bit off more than I could chew. So I've gotten better at being more intentional about that and knowing you can't just slam your head against a brick wall. You have to find ways to break the problem down, to get a sense of getting a small win, and then you'll just come back tomorrow and find another small win, and they will build over time, and that's really essential. So the blockers were myriad. It started with my foundational goal is I want a 100% compatible parser, and one of the things I realized as I did some initial research is that, as far as I know, there is not a single 100% compatible SQLite parser in any language, in any ecosystem. There are roughly two types of parsers that I find. So one is a generic SQL parser. It's like I'm going to parse everything for SQLite, mysql, Postgres, and in order to find the union of those different dialects, they drop some things that are specific. Or you have SQLite-specific parsers and for those what I found is that they drove their implementation based on SQLite's public documentation, and SQLite's public documentation is really excellent. They have really useful, really beautiful syntax diagrams. They break down the syntax, but those syntax diagrams I have come to learn, having written a parser with a focus on compatibility are much more so diagrams for how to properly write SQL than how the parser actually parses SQL.
Speaker 2:There's more flexibility in the parser than there is in the syntax diagrams and so, just as one really straightforward example, when you're defining your table with a create table statement, you can define a column and you can put some constraints on that column, right. So create table foo, open parens. Your first column's name is bar and you can say not null, that's a column constraint. You could add a primary key constraint, a foreign key constraint, a check constraint, whatever you want you can. Also, at the end of your list of columns, you can start specifying table constraints so you can say like oh, I'm going to define the primary key.
Speaker 2:I need a composite primary key so I can't just attach that constraint to one column. I need a table constraint here at the end. And you can do many of the same constraints primary key, foreign key, references, check constraints just at the level of the table. And the syntax diagrams will tell you that those different table constraints are separated by commas. And that's the only thing that you see in the syntax diagram and every parser I've looked at that's what they encode. Sqlite's actual parser that actually runs in your programs is like it's fine, you can separate them with spaces, I'll figure it out.
Speaker 1:Okay.
Speaker 2:So there's valid SQLite SQL that you can pass into the CLI, you can send from your Rails app and it'll work totally fine and most parsers will throw an error. I mean, every parser I've researched would throw an error if you used a space to separate table constraints. And there are others like that. So that was like the first big blocker was like just to realize that I couldn't just go off of the public documentation and that I couldn't just look at other parsers of which there are a handful even just focus on SQLite, that I was going to have to find some way to make sense of the C implementation. And just to add, why do any of this at all whatsoever? There's not a great answer, I'll be honest, but part of it is that, unlike Postgres, Postgres has a really beautiful aspect of its implementation in that it has a publicly exposed independent parser. So Postgres' own parser that the engine uses is available and you can write your own binding. So PGQuery, the gem that many of your listeners I'm sure have heard of, if not used, has Ruby bindings into Postgres' parser and there has been a whole beautiful ecosystem of tools that have built on top of that, knowing that the way that they see the SQL is exactly the way that Postgres will see the SQL and that allows for all kinds of interesting functionality.
Speaker 2:That's hard to predict when you first do that. Part of what makes SQLite so fast is that its parser does not parse your queries into an AST and then take the AST and execute the bytecode instructions for the SQLite engine. It skips the AST part. It says I go from taking your string, interpreting it by my grammar, immediately into bytecode instructions that I immediately execute. There's no intermediate layer, so you can't do anything with SQLite's parser. It's completely and fully integrated into its own virtual machine. So there's no way to ask SQLite like how do you see this? It's like it just executes it immediately.
Speaker 1:Great for performance, not so great for building tools on top of Exactly.
Speaker 2:I have all these different ideas for like so many tools that would be so cool, but they require being able to understand the SQL in the same way that SQLite understands it. So yeah, that barrier of like. Okay, I'm going to commit myself to 100% compatibility. The documentation doesn't perfectly encode the actual grammar. It encodes the idealized version of the grammar. So I'm going to have to dig into the C code and make sense of what's going on here and figure out how to write good tests and to get to 100% compatibility. And then, of course, the other huge blocker which we could just talk about forever it would be boring and so we won't is learning basically how to write a recursive descent parser by hand in a way that is performant and maintainable and correct. And there's a lot of iteration and reading and experimentation. But I ended up in a place I'm really happy with and the code is public source. It's a project called Plume you can find on my GitHub.
Speaker 1:There'll be links to that. It is very interesting the uniqueness of the problems and blockers that you run into this with right. It's not just like the intrinsic complexity of writing a parser, it's also this documentation issue where you can't just look at the documentation. You actually have to look at how sequel light is doing this under the hood. You don't get an astST to reference. So that is an undertaking, sir.
Speaker 2:Sometimes it's better to be foolish than wise.
Speaker 1:Sure, and it's interesting because a lot of what you said was, I remember, when we had Kevin Newton on the show oh my gosh, that was forever ago, but Prism was fairly new, but he was kind of had the same feelings of you dudes Like we need an AST in order to build tools. Like we can't just go straight to bytecode. We lose the ability to actually look at the code we're writing and do any sort of error-tolerant analysis of Ruby. If we want to have the same tools that we get in other languages, we need an AST and we need a parser that'll generate that for us and that's hand wrote a recursive descent parser for Ruby. That was the whole motivation and I mean I've worked a little bit on Prism with them and good luck to you, sir Godspeed.
Speaker 2:Well, I will say that I've learned a lot of lessons around, like where to apply some leverage. And the endeavor to build Prism is a wild and ambitious endeavor and it's really, really amazing that they did such a good job of it, because they did it in hard mode, like the fullest of hard modes, because you can sort of layer on technical challenges to writing a parser, because you can sort of layer on technical challenges to writing a parser, and so one level you can add is to say, all right, I'm going to handwrite a parser and not just use a parser generator off the shelf. If you just use a parser generator, really the only complexity is you have to really encode your grammar correctly and well, but you just basically just think about grammars, which is easier. Now the pain you have is that, as things need to change, you can't change anything in the actual parser and you can't get any optimizations at the actual parser. You can just only change the grammar. So if you say, all right, I'm going to handwrite it, that'll allow me to do specific optimization, specific features, that's harder.
Speaker 2:Then if you come up and you say, okay, I need to define like my AST, like what are the names of things. What are the things like? It just takes time and you have to figure it out and how you do that consistently. That's hard. But then you come up and you say how concrete does my syntax tree need to be Right, like most of the time we just say abstract syntax tree. That's why we all say it.
Speaker 2:There are options. You can have an abstract syntax tree or a concrete syntax tree, where the difference there is effectively simply can you perfectly reproduce the input string from the syntax tree? If you can't, then it's abstract, and if you can, then it's concrete syntax tree. If you can't, then it's abstract, and if you can, then it's concrete. And if you say I want to have full understanding of the input string, so I'm going to assign semantic and syntactic meaning to every character in here, you're just going to have to keep track of everything and name everything. That's already harder, which is what Prism does. And then if you say my parser is going to be fault tolerant, so regardless of the string you pass it, you will get back a syntax tree and you can introspect that that's harder. And then if you say I want that syntax tree to be maximally informative, so I'm not just going to, instead of raising an error at this character, I'll just add a little node that says, hey, error encountered here. And that's your node.
Speaker 2:You try to isolate those things and do heuristics to allow as rich and as full a syntax tree as possible to be built. That's quite hard. And then if you want to do all of that and you want to do it incredibly fast, that's quite hard. And that's what they did in Prism and all I am trying to do is just parse fully completed strings, and I'm not even trying to do it incredibly quickly. I have an eye towards performance, but I'm writing it in Ruby and I'm not writing it in C.
Speaker 2:So, just to call out, as someone who's now gotten into it, I have such a deeper appreciation for the Prism project. It was a daunting task. It took that team years and it is really, really astounding that they did such a good job of it. And the point that we have been talking about just to bring it all the way back now. It opens up so many possibilities and it is impossible to predict what it will allow us to do in Ruby with Ruby, because we now have this full, useful, concrete, fault-tolerant parser, concrete, fault-tolerant parser. Now we just need to give people time for their imaginations to rev up Such an important project in the Ruby community.
Speaker 1:You made the reference of, because Postgres has this parser that makes the AST for you. We have all of these tools built on top of that parser that makes Postgres that much funner to work with as a database or easier or more secure, or what have you. We don't get that with SQLite because we don't have that parser. That's sort of what you're going for and might not be as robust or concrete or performant perhaps at least in the beginning of it as Prism, but the idea, what it unlocks for us, is the same. Is that reasonable?
Speaker 2:Yeah, essential difference that took me some time to fully come to terms with is that Prism's fault tolerance and performance are aimed at making it a tool that can be used in live text editing, and that's a feature set that I'm not interested in supporting. I'm not going to write a parser in Ruby to help your Vim setup have better syntax highlighting as you're typing SQL. That's just not where I'm at with my life. I'm crazy, but I'm not that crazy.
Speaker 1:You just got to nerd snipe someone else on that front.
Speaker 2:But at that level of you have full sql statement. So we take the example from rails, right, like the schema is there, you can ask the database what are the create table statements and it'll return them to you. They're full and complete and valid. That's a given. And now you need to understand everything about that schema. Now that will be possible. Like in rails, it understands as much as it cares to understand, as correctly as it kind of can, but there are limitations. You can sort of think of it as it's basically the inverse of Errol. With Errol you can go from Ruby to SQL and with Plume you can go from SQL to Ruby.
Speaker 2:And one of the primary larger goals with the project is to build basically an alternative to Arrow to allow you to encode your SQL in Ruby in a way that feels like good, beautiful Ruby, but is as expressive as SQL, expressive as SQL. This is one of the frustrations with active record. Active record is great until it isn't and you're like, well, I kind of need a left outer join, I kind of need an inner join. It's like, well, just write a string inside of the joins command. Or like, build out this 10 line arrow method where you're like you're getting the correct table references and you're getting the correct column references and you're calling infix operator methods to set up all of these things. It doesn't feel like beautiful Ruby. So for me, that's the layer I'm definitely going to build on top of. It is a really expressive layer Ruby syntax to generate SQL that is as expressive as SQL. Right, like there's no limitations, and then so many other possibilities.
Speaker 1:That is awesome. Not jealous of the work that you're doing. I'm looking forward to reaping all of the benefits from the work that you're doing. So thank you. But man, I'm not jealous of any of it. It's a very large undertaking. Are you hoping, planning to have help to get like people on board with the? Hey, there's a bunch of sequel light nerds out there who also love ruby. Let's work on this together, are you kind of like? I'm happy doing it on my own, as my own little project and maybe someday in the future.
Speaker 2:I don't expect anyone else to get to full-blown maintainer level.
Speaker 2:I don't know how many SQL nerds there are in Ruby who are like I want to download the entirety of the correct grammar of SQLite into my brain.
Speaker 2:It's a stupid thing to do, but I pair with Joel Draper basically every day.
Speaker 2:We work on all of our open source projects together and we talk through a lot of the architectural decisions and where the features and how we want the interface to be and how we want to name things, and I do hope that people will get inspired to build tools on top of it. But my general hope and idea is that I will go off into a cave and read all of the C code and name all of the stuff and come back and say, okay, you give me SQL, I give you back a Ruby AST and I make the promise to you that if you can pass that SQL into the SQLite CLI correctly, you will get back an AST and that AST will match how SQLite understands that string. And so I really do hope to get bug reports for edge cases where the parser is wrong and I hope that people build some interesting tools on top of it. And if there are any other sickos out there who love writing parsers in Ruby and love SQLite's dialect of SQL, hit me up, but I don't expect there to be.
Speaker 1:Oh, it's a bit niche, but you never know, there's all kinds of developers in the world and never know who you might run into. So all that brings us to our last question, bit of a loaded one for you. What is something cool, new or interesting that you've recently learned, discovered, built? It doesn't have to be code related. Yeah, very well, maybe with you, but what is something cool, new or interesting from you?
Speaker 2:I'm going to cheat and we're going to give two answers. Go for it. The first one is it won't take up enough time. But I saw something really interesting recently online. I was at highleveragerailscom and it was like whoa okay. All of a sudden my eyes were opened and the heavens parted and I realized that it would be possible for me to learn how to build high quality Rails applications by mastering only two tools Rails and SQLite. It's kind of amazing to think that you can learn once but then ship fast and ship quality. What a bargain. What a bargain. I would check that out. What a pitch. What a bargain. I would check that out.
Speaker 1:What a pitch, very nice. I liked how you worked that in there.
Speaker 2:Sometimes I try to pull out my dad's radio voice. My dad was a radio DJ for years, was he now? Yeah, that's awesome, hi leveragerailscom.
Speaker 1:You do a great job with it. It's like a different person that we're talking to. It's fantastic.
Speaker 2:That really is cool and exciting. But, on a somewhat more serious note, I have been using in this new parser project one of Joel Draper's newer gems, which is a gem called Quickdraw, and Quickdraw is an alternative to RSpec and Minitest. It is an alternative testing framework, and the particular killer feature behind it is that it is a testing framework designed for our modern multi-core environment, and so it will run your tests using as many cores as your computer has slash, as you want it to use, and so it is objectively the fastest testing framework in the Ruby ecosystem. It has many tests compatibility. If you want to keep using many test assertions. It has our spec compatibility. It has its own assertion library.
Speaker 2:Especially for a project like this, where I went and scraped every single SQL statement in all of SQLite's public repository, of which there are about 20,000 unique SQL statements, and I test each of them that I can parse them, and eventually I'm going to do a parse generate parse pipeline to make sure you get out what you put in, and then I have 10,000 of my own tests, and so being able to run 30,000 tests in 600 milliseconds so just put the tests on watch mode and push forward has been a major productivity boost and, as I said, joel and I pair on our open source work basically every day.
Speaker 2:It's been cool to see that project come together and on every team I've been on, at every company I've worked on, it is a consistent cry. Ci is too slow and one of the things that we realized as he first sort of came up with the idea and the thought is, if you look at a lot of Ruby gems, like they tend to have pretty fast test suites, like it's not uncommon to have 10,000 tests and have them run in less than a second if you just have a gem. But in Rails applications it's like 20 minutes to run your tests and really embracing the power that our modern machines have and he spent a lot of time on the work-stealing algorithm to make it as efficient as possible it will saturate your machine. It'll run those tests as quickly as possible, including your slower IO-bound tests. So I think it could be quite useful for teams to check out and I've really enjoyed using it the last month or so.
Speaker 1:Yeah, that is definitely one that I'll have to check out, and you said it's compatible with both Minitest and RSpec. It has its own assertion syntax. That sounds really cool. Joltrapper is another code encoder. He's been on. He was on a while ago. I might have to have him back on. He's always busy with something. He's always busy. That is cool. You have quite a bit on your plate. I'm super excited for the course, excited to see where your new parser takes you and takes SQLite and all the goodies that could come out of that. But definitely want to have you back on when you're getting to a point where you have more to talk about on it or anything else that you nerd snipe yourself on and you want to come back on the show to talk about.
Speaker 1:You're always an interesting person to talk about. Always good answers to all three. So, as a wrap up, where can people find you and your course online?
Speaker 2:Yeah, be sure to check out highleveragerailscom. If not there, you can find me on all of the social media platforms. At Fractal Mind, feel free to reach out and I will. Small extra plug here at the very end. Talking with Drew, I have to plug it For those who might not have heard RailsConf this year will be the only Ruby Central Conference in the US and it will be hosted in Philadelphia. For those who don't know, I lived in Philadelphia for eight years before I moved to Berlin. I stayed up till four in the morning in Berlin watching the AFC and NFC championship games, rooting for the Eagles fly, eagles fly, and I'm incredibly excited for RailsConf and you should absolutely get the early bird tickets as soon as they come on sale. It's going to be a banger conference. This is the last RailsConf. I'll let Drew add to the pitch in just a bit, but it's got to be said if we have two Philly boys on the call A, fly Eagles, fly B RailsConf in Philly 2025. These are important things to say.
Speaker 1:Yes, sir, yeah, railsconf in Philly. The first time Ruby Central is doing any of their conferences in Philadelphia. It will be the last RailsConf. There's so much that I don't think I'm allowed to say in public yet. So I'll just say I know that we have awesome co-chairs. The programming committee is shaping up to be incredible. It is going to be an awesome conference. It's going to be a great party. As soon as we drop the early bird tickets, I highly recommend getting on there. Don't wait, so that we can make sure that we have a good headcount, so that we can make it as awesome as humanly possible. Definitely good shout out there. I'm excited, so excited to have everyone come to Philly and experience our food and all the history here and have an awesome time at RailsConf. Thanks for coming on, thanks for talking about everything you have going on and thanks for the shout out for RailsConf and we will have you on again soon, for sure. Yeah, thank you, it was a lot of fun, as always. See you all in the next episode.
