This transcript was generated automatically by AI. If you find any mistakes, please email us.
0:00:00
(Ganesh)
Hello, listeners and viewers. Today's session is actually both a live webinar and a special recording of the Cloud Next podcast rolled into one. So whether you're here with us now, catching this later on Spotify or Apple or watching on YouTube, welcome.
0:00:16
(Ganesh)
I'm Ganesh The Awesome, a Solutions Architect and Sales Engineer here at GlobalDots. And I want to start by saying, well done you. You made a very wise investment of your time by joining us today. Because when it comes to scaling, let's be honest,
0:00:29
(Ganesh)
it's where startups go to die. But don't worry, over the next 45 minutes, we're giving you the shortcuts, real war stories with zero fluff. You'll learn how to spot the early signs of scaling stress, avoid infrastructure decisions
0:00:42
(Ganesh)
that'll haunt you a year from now, and level up your Git game even if you're currently limited to pull, branch, merge, commit, dot, dot, dot, panic. These aren't just tips, these are battle-tested solutions from pros who've been through the fire and made it out with their code and their budgets intact. Quick intro, who are we at GlobalDots? We help high growth companies scale without breaking. For over 20 years, we've been building resilient infrastructure, securing cloud environments,
0:01:10
(Ganesh)
optimizing costs and prepping startups for the serious stuff like SOC 2, compliance and real world production traffic. I'm super pleased to be joined by two people much smarter than me and much more experienced than me. And they'll be guiding us through today's discussion. Aded Saghi from Aqquant, who has years of hands-on experience, scaling infrastructure for high-growth startups
0:01:33
(Ganesh)
and enterprise platforms. Aded, would you like to introduce yourself?
0:01:37
(Oded)
Yeah, thanks. Thanks, Dinesh. And hi, everyone. Happy to join today and tell a little bit about our decisions, our story here in Aquant. I'm leading the product and R&D teams
0:01:51
(Oded)
for the last few years in Aquant. I'm coming with a background in the software industry when AI and machine learning wasn't cool at all, building that for SAS enterprises, building that for SAS enterprises, building that for SAS startups, working a significant amount of time in hybrid,
0:02:11
(Oded)
multidisciplinary companies that produce equipment as well. So hope it's gonna be interesting today.
0:02:20
(Ganesh)
Great, thanks Odette. And our other speaker today is Nir Gaya, AKA the King of Git, who brings deep expertise in advanced Git techniques. Nir, would you like to introduce yourself?
0:02:32
(Nir)
Yes, I will do it during the lecture as well. I'm Nir, I'm talking, I will talk about Git, how we can enhance Git and improve our productivity with a lot of hidden features. And we will meet you later on.
0:02:45
(Ganesh)
Great, thanks Nir. I'm 100% sure that I'm going to learn things from that session as my Git skills are definitely somewhat limited. Now, both Odette and Nir have worked with many organizations at different stages, from lean startup to hypergrowth. And they'll be sharing their real-world insights with us
0:03:03
(Ganesh)
today, from traveling through those organizational bottlenecks that lots of companies hit shortly after raising their Series A. So let's set the stage before we dive into the tactics and takeaways.
0:03:14
(Ganesh)
For many of you, you've recently raised your Series A, or you're in the process of scaling out your teams and products, and you're preparing for the next phase. This is what we call our series A inflection point. You've proven the product market fit, so well done you. But now comes the next hurdle, which is making
0:03:29
(Ganesh)
sure your infrastructure, processes and teams can scale at the same pace as your customer demand. And here's the thing, it's not just a technical challenge, it's a strategic one. Without the right preparation, growth can actually become a liability and I'm regularly meeting startups in this phase here at GlobalDots. And we routinely see a few things crop up.
0:03:52
(Ganesh)
First, growing fast without foundational readiness. Your code base is shipping, your features are rolling out, customers are coming on board, but your backend, your CICD, your observability and your compliance, possibly a step behind, maybe.
0:04:06
(Ganesh)
Secondly, systems that are scattered and stitched together, stitched together, reactively, which makes them fragile, hard to debug, and increasingly expensive to manage. And third, a difficulty in hiring
0:04:20
(Ganesh)
senior experienced DevOps talent. And as we well know, the best people are hard to find, usually because Google and Meta has gobbled them up. And even if you do find them, it can take months to onboard them and integrate them into your team.
0:04:33
(Ganesh)
So how do you avoid these dreary pitfalls without slowing down your innovation? That's exactly what we're diving into today. So as a CTO or a VP of engineering or a product leader, your strategic priorities at this point should boil down to building resilient
0:04:49
(Ganesh)
and automated infrastructure, but without needing to stand up an entire in-house DevOps team from scratch, maintaining visibility and cost controls while keeping your release velocity high, and perhaps most importantly of all,
0:05:01
(Ganesh)
making the right architectural decisions now to avoid massive and expensive rebuilds a year from now. One approach we'll be exploring today is DAS, and that's not disaster as a service, but DevOps as a service. And this emerging model gives you immediate ability
0:05:18
(Ganesh)
to add DevOps expertise and get faster execution without the overhead of building everything internally. You can think of it as plug and play DevOps
0:05:27
(Speaker 9)
that grows with you.
0:05:29
(Ganesh)
And whether you're wondering what to prioritize over the next six months, or you've already seen the signs of technical debt catching up with you, today's session will give you real life tangible strategies that you can start applying today.
0:05:40
(Ganesh)
By the end of this webinar, we hope to give you clarity on identifying your biggest scaling risks before they turn into blockers, understanding what to build in-house and what to outsource, leverage DevOps as a service to scale smarter and faster,
0:05:53
(Ganesh)
how to be a total ninja with Git, showing you all the tricks you wish you knew already today, and help you make decisions that should save you months or even years of time down the
0:06:06
(Speaker 8)
road.
0:06:08
(Ganesh)
So that's it. Let's jump in. Oded, why don't you kick us off with a little bit about what you've seen in your work scaling startups.
0:06:15
(Oded)
Great. Hey, everyone. Thanks again for taking the time today. I'm Oded Sagil, I mentioned VP of products and R&D over at Aquant. Really thankful for the opportunity to organize my thoughts for this webinar. You know, since our early funding days, we've been in constant growth here in Aquant.
0:06:40
(Oded)
We introduced more products, we introduced more personas to our product, we grew our R&D team and our software deployments within Fortune 100 companies. And being completely honest, and you probably imagine it hasn't been so smooth, right? We all had late night fire drills, we all had cracking migrations and surprise outages. So what I try to do today is to share the stuff I wish I'd known before that growth. A little bit about how I think, my thought process and how my decision making process looks like. And these are maybe lessons you already know
0:07:19
(Oded)
but hopefully you can learn some of it and use it before, you know, you learn it the hard way, like me. So a little bit about the Quant. Quant is a platform for service team. We're helping OEMs, we're helping manufacturers or field service organizations, call centers to resolve customer issues faster,
0:07:41
(Oded)
and with your repeat failures, okay? So we support a wide range of touchpoints like mobile apps, web interfaces, APIs, integration within systems like Salesforce, Microsoft, and we've actually been doing that way before AI became the mainstream. But over time, we built the data infrastructure to handle the reliability, the mainstream, but over time we built the data infrastructure to handle
0:08:05
(Oded)
the reliability, the security and reliability required in our space. And so as you can see, we operate across multiple industries, from medical devices and heavy machinery to manufacturing, printing, building services. We are trusted by global leaders like Siemens, Coke, John Deere, and we're also really fortunate to be backed up by some of the respected investors out there like Lightspeed and Insight. Now let me show you a little bit about how this growth that I mentioned is translated
0:08:36
(Oded)
into scale and into complexity and later to complexity behind the scenes. This is some figures about 2024, which was a great year of usage growth. So more than 3000 growth in interactions really means that across all channels, mobile, voice, API, you name it, we grew our volumes. And what we call our useful rate, which tells us how our recommendations
0:09:04
(Oded)
that we generate are actually being used and not ignored. And so we also saw more and more product metrics that associated with growth like DAO and MAU. That means that our users are coming back to and engaging with the systems and like they're not trying it off and dropping. And I shared a little bit more key figures here. You probably noticed that we tried to maintain the five, nine level of availability.
0:09:31
(Oded)
We really try hard to keep this uptime, which at scale isn't given. So with that kind of growth and the expectations that we have from our customers, we needed to make real changes under the hood. But before I go into these changes,
0:09:53
(Oded)
I want to take you to the early stages back when we were just getting started. We had what I call pretty classic founders-led mindset. You all know that. It's about moving fast. It's about don't overthink in front, no DevOps.
0:10:13
(Oded)
Definitely don't build anything you don't absolutely need to. That's why we went all in on Heroku. Heroku is a platform as a service. Under the hood, it's AWS, but it's owned by Salesforce. It was simple. It was fast.
0:10:30
(Oded)
It gave us just enough flexibility to deploy Postgres, S3 core logics as plug and play add-ons. No heavy lifting, no real ops, just ship. And this, by the way, this wasn't just a technology decision. It made sense for us from go-to-market perspective too.
0:10:50
(Oded)
We needed to spin up instance demo, we need to make customizations, we need to deploy changes fast and Heroku just let us do that. It fit perfectly also with our strategic ambitious to partner with Salesforce.
0:11:04
(Oded)
Now at this point, we're just a few developers. We're pushing code 10 times a week. Our CI-CD pipeline was literally Git push. No feature flags, it would just rows in database, no infra team. That image that I'm showing here in this slide is
0:11:22
(Oded)
a cartoon of a developer wearing a Roku hoodie. That's more or less exactly how we felt at that time. By the way, they never sent us a hoodie. For a while, it worked great. But of course, we knew it wouldn't last forever. But we made a deliberate tradeoff, okay? Speed over control. And honestly, it was the right call for where we were back then. Infrastructure wasn't our differentiator. It was about the velocity.
0:11:57
(Oded)
At some point, things started cracking, okay? Because we weren't just building features anymore. We were running data pipelines, we were training machine learning models, we were building the foundation for API and authentication layers.
0:12:10
(Oded)
So our info cycle just couldn't keep up. I'm talking about cold starts, timeout, slug size limitations, things that directly impacted our reliability and our ability to deliver software.
0:12:27
(Oded)
Like to stay in control, we had to build custom stuff like auto scalers and tune them constantly. It literally ate up time and engineering focus. And I'm a developer in my roots, okay? And you guys all know about developers, they will find the easiest path to resolution.
0:12:47
(Oded)
And in our case, team started to drift. If Heroku was not enough, let's go with EC2, right? And when needed, let's scale to EKS. Why not? Why not spinning up a VM on Azure? So our infirm was practically duct taped together.
0:13:05
(Oded)
We use telephone here, we use scripts there. It wasn't long before the moral and the overall developer experience took a hit. Now that image is our kind of Darwin evolution of developers juggling cloud components just to keep things going.
0:13:26
(Oded)
Now, you can't actually see our frustration, but believe me, it was there. Okay? We're not smiling all the time. Because observability was also a mess. You know, we had logs scatters everywhere. We didn't have a unified view. So debugging took days.
0:13:42
(Oded)
Now imagine how brutal it was to achieve SOC2 certificates, ESO, how to go through pen tests. It was tough, but the kicker was we were still a startup on a credit card custom mail with those cloud vendors. So we were having no discounts, we having no support.
0:14:04
(Oded)
And as the usage grew, so did the costs, right? And we couldn't optimize fast enough. And so the big takeaway here was that speed was the right call early on. But if you didn't design for scale, the quick wins will turn into long-term debt with interest.
0:14:26
(Oded)
That's more or less was the moment where we had to make a real decision on what to do with our scattered Cloud infrastructure. We knew the current state wasn't sustainable, and we wanted to consolidate, probably through migration process.
0:14:41
(Oded)
But every option on the table came with trade-offs. So what you're looking at here now is a high level presentation of our decision metrics. The vertical axis, the TDR, like the technical depth ratio. So the higher up you are, the more legacy pain you're carrying forward.
0:15:01
(Oded)
The horizontal axis is the business risk, is the business risk, sorry, things like time to value, team destruction, production risk. So let's start at the top left. Stay, don't do anything, lowest business risk, right? But tons of technical debt,
0:15:18
(Oded)
basically keep doing what we're doing and accept the pain. But this was not a real option, right?
0:15:45
(Speaker 8)
which may actually increase the risk.
0:15:51
(Oded)
On the bottom right, there was the self-rewrite option, kind of the self-migration piece. Open a clean state, right? Take full control, but the risk is high, right? It would have taken us months and would have most definitely distractedructed the small team. It would require headcount that we didn't have.
0:16:08
(Oded)
And honestly, we didn't know how to recruit it even. So too slow, too expensive, too fragile for where we were. And so we went bottom left at what I call partner augmented migration. Kind of giving us the best of both worlds, low risk, meaningful technical progress.
0:16:34
(Oded)
And so, choosing a partner augmented migration, or just like Ganesh was mentioning, DAS, that route made tons of sense for us.
0:16:45
(Speaker 9)
We didn't need another small improvement, right? Thus, that route made tons of sense for us.
0:16:47
(Oded)
We didn't need another small improvement, right? We needed a real step change, something that could actually move the needle. So we ended up bringing a senior DevOps folks and honestly, they were up and running within hours. No hiring process on my end, no onboarding delays, no pulling our team off the work,
0:17:06
(Oded)
it just keeping the immediate momentum. It wasn't about outsourcing, right? It was highly focused and high impact help. While our internal team kept driving the product and the roadmap, the partner team handled the heavy lifting with designated individuals from the organic team
0:17:27
(Oded)
for specific areas. Now, because that partner matched our pace, our mindset, our quality standards, it really felt like they were part of the team, like an extension to the team, and not some external add-on that we had to manage.
0:17:45
(Oded)
The takeaway from this experience is that if you're aiming for real transformation, the real value would be in bringing experts to drive that change. Don't bring expert just to patch your day-to-day gaps. I want to tell you a little bit about that migration. And here was the plan. This was the plan.
0:18:08
(Oded)
You see this neat, super well-structured 12-week migration plan, phase rollout, everything scoped, designed, and all the way to go live. Looks great. Honestly, we were proud of how clean it was. And then reality happens, right?
0:18:29
(Oded)
Pretty quickly, we realized the nice linear plan wasn't going to fly. We had to toss most of it, really, and start small. We picked one up, we modernized it, we moved it over to AKS, to Azure. We shifted the storage to Blob, we migrated the databases. But we also used traffic shadowing to safely test it under real considerations without breaking for the actual users.
0:18:55
(Oded)
Now, we did it for every app, although it had its own architecture, but we stuck to the same basic playbook. We ran parallel environment side-by-side, used proxy stuck to the same basic playbook. We ran parallel environments side by side, used proxy service to route traffic, gave us tons of flexibility, really,
0:19:11
(Oded)
and let us roll things out without going in too early. It was a messy at start, but we didn't skip steps. We just kept reprioritizing based on what was actually breaking or blocking go live. Now, honestly thinking, I don't think this is a failure that, you know, the original plan shifted. The plan gave us direction for sure, but the execution needed the flexibility, the fast feedback loops, and we needed a lot of creative problem solving that our partner brought to the table.
0:19:49
(Oded)
So one geeky slide, I'm sure, but I want to share that after all the chaos and iteration during the migration, this is where we ended up. We had a modern production grade cloud stack. It finally lines up, you know, with how we want to build, how we want to scale and operate. Now this diagram is a very high level one.
0:20:06
(Oded)
It shows more or less the full stack from VPN and app gateways all the way down to the message queues, storage layers, AI workloads. It's pretty dense layout, but every part was designed to scale cleanly and run efficiently. And so the key elements of this change,
0:20:27
(Oded)
and sorry again for getting into the geek talk, is about containerizing our Python and Celery workloads and move everything over to Azure AKS to give us the consistent and scalable foundation, especially with reusable hand charts and KEDA-based auto-scaling baked in from the get go.
0:20:47
(Oded)
And now everything was built with infrastructure as code mindset. We use Terraform to provision the AKS clusters, registries, key vaults, CloudFlare configs that gave us the security, the auditing level, the predictability that we were really missing before.
0:21:05
(Oded)
Observability, which is a big deal. We rolled out ground cover, okay? Which this gave us full visibility into logs, metrics, traces, but the upside of that for me as well is the developers don't have to manually instrument anything. For CI, CD, we moved fully into GitHub Actions,
0:21:27
(Oded)
you know, automating everything, builds, tests, security scans, helm deployments, telephone workflows. Now, it wasn't just building the pipeline, it was also to coach our teams, internal teams on how to get the most out of Kubernetes.
0:21:44
(Oded)
And of course, one of the most important aspects that we couldn't ignore cost. It was spiking up at the beginning. So early on, we brought in Carpenter and later we signed with Cust AI to handle the auto-scaling intelligently
0:22:00
(Oded)
and actually match the resource we use to actual load and our budget goals. So this was our end of a very practical, not theory of evolution, practical evolution process. And this is what I want to leave with you all today. Moving fast, early is essential,
0:22:22
(Oded)
but every shortcut accumulates tech debt and must later be repaid. Pay attention to the flags like rising latency, platform limits, escalating cloud spend. These are all flags that speed is now stalling the scale. The decision matrix or the thought process, stay or lift and shift lift and shift, you know, don't preserve the pain. That's the bottom line.
0:22:49
(Oded)
And at certain situation, a partner augmented path can balance the velocity, the risk and capability that you're looking for. Plans are awesome. But you know, if you're doing migration, migrate one service, shadow the live traffic,
0:23:05
(Oded)
validate the performance, do all of that before rolling that across your entire stack. Yeah, external experts will accelerate the transformational work while internal teams can stay focused on the roadmap, and modernize around the principles
0:23:25
(Oded)
of containerization, infrastructural code, unified observability, et cetera. And that would be the outcome, the secured, observable, cost-efficient architecture that you need, that you know that now grows with your business, free your engineers to deliver and not firefight.
0:23:51
(Oded)
So thank you all. And if this journey resonates with you or you're just about to start one, or you just want to chat, scan the QR code to connect over LinkedIn, happy to swap war stories,
0:24:06
(Oded)
or even set up, you know, 15 to 30 minutes, one hour to review. Appreciate the time. Back to you, Ganesh.
0:24:16
(Ganesh)
Pleasure, Oded, thanks. And that is honestly a textbook story of a great collaboration with DevOps in there. I wrote down, the quick wins will turn into long-term debts with interest. I think that's like the quote that I took from that presentation. But yeah, it's interesting because people think of that DevOps as an outsourcing,
0:24:40
(Ganesh)
but I don't think it is an outsourcing. It's an insourcing because you're bringing them into your team to build your environment. You're not handing away the problem. You're bringing someone in to help with your problem. And you mentioned some very cool technologies in there, and specifically Groundcover as an open telemetry
0:24:58
(Ganesh)
and a kit that comes with Victoria Metrics and Clickhouse, all in one that's inside your estate, is an unbelievably great way to cost optimize. So you shared some like really magic tools that helped you along your journey there. And yeah, we see lots of people in that startup stage
0:25:15
(Ganesh)
who are living in the Heroku world. And yeah, thank you for sharing those steps and your pain. So maybe other people can avoid that. Really great, appreciate that. Let's shift gears a little bit. If you've ever spent hours resolving the same merge conflict
0:25:30
(Ganesh)
or waited forever for a massive repo to load or just felt like Git is a mysterious black box, Nir's about to change that. Nir, take us away.
0:25:39
(Nir)
Okay, I'm sharing the screen. Cool. Okay, my name is Nir. I'm going to introduce myself. You have the QR code to scan if you want to take the slides. I will put it at the end as well. If you did not get it up, you will have it later on. I'm starting and then I will introduce myself
0:25:54
(Nir)
as well. Once upon a time, as we say, there was 2007. In 2007, a few things happened. First of all, the first iPhone, which is not relevant to what we're going to talk about. And I'm going to tell you a little story. What happened to me in 2007. In 2007, I was a CTO of a small company. You were something like eight people in the company.
0:26:16
(Nir)
I decided to take the existing code that we have and to improve the building. Again, it's 2007, DevOps was not as common as it was today. And I started to work with something called Maven. It's not relevant, but I wanted to improve the build. We worked with the source control called CVS.
0:26:35
(Nir)
Before I started to work, I did a backup. I backed up all the files of the source control and starting to work, improving my automation. After a while, I ran out of space. I ran out of space. I searched for a few big files.
0:26:49
(Nir)
I found two big files in my file system and I deleted them. After I deleted them, I continue working. And instead of deleting the temp folder, accidentally I deleted the CVS, the source control. Oops, what are you doing now?
0:27:08
(Nir)
What I did, I went to the backups. I was searching for the backups and oops, no backup because I deleted them. I said, oops again. I went to the, I tried to recover it. I couldn't find it obviously.
0:27:22
(Nir)
So I went to the IT guy. I told him, listen, I did some problem. Can we recover what they did? We tried to recover the file, but it was company that worked with credit card and then money and so on.
0:27:35
(Nir)
So it's obviously, it did a wipe zero, which means that no data can be recovered. Oops, for the third time. I said, nevermind, let's go to the backups. We wanted to I wanted to see if there was a backup and they worked on the code for about two years prior to me running to the company and no backup. Oops, three days later I was sitting in the office I took some
0:27:59
(Nir)
some code some content from a few people who worked with me. I tried to recover what I could, but I couldn't. Let's be honest. I don't know how much I did recover. I can estimate about 50%, not more than that. And I said, never again. Since then, as I mentioned,
0:28:17
(Nir)
we worked with something called CVS. I went to move to work with Git because it cannot happen in Git. Again, we don't have a lot of time today to talk just about Git. I'm going to show you a few things
0:28:28
(Nir)
which can improve your quality working with Git. And since 2007, I started working with Git. I did a lot of Git project. I contributed to a lot of open source related to Git, to Git itself, to some other repositories as well. I'm a well-known user on Stack Overflow.
0:28:47
(Nir)
In Stack Overflow, I help for more than 50 million people, which is incredible. I'm answering questions. Once the question, those are two of my questions. Again, I have many of them. As you can see, I answered almost 2,000 questions, close to that. What I'm going to talk about today,
0:29:07
(Nir)
I'm going to talk about a few things in Git, internally in Git, which people are not aware of, and they can improve your productivity with very, very simple steps that you can follow. Everything is in the slides. We're going to show live demos as well.
0:29:22
(Nir)
So you have all the code and everything again. As I mentioned, Git was started in 2005. And since then, a lot of Git has evolved. The man who involved Git is Linus Tovalds. He named Git after words in English, sorry, after words in English, which is named Git,
0:29:40
(Nir)
which is describe what is. There are a few slides about Git. If someone is not familiar with Git, I'm not going to read them. You can follow them later on. I want to focus on the main issue that we have.
0:29:52
(Nir)
So usually developers or participants are hating Git. Why do they hate Git? Because Git is very, very complex. They are coming from another source control like CVS, TFS, SVN, whatever, and someone is telling them, listen, we're not going to switch to Git,
0:30:08
(Nir)
and it's not as simple as it sounds. Git is different from many, many other source controls. We don't have time today to go over all the things, but we're going to show a few things which can, as I mentioned, can improve your productivity.
0:30:21
(Nir)
For example, I'm giving a very tiny example why Git is so complex. In Git, if you want to rename file, if you simply name it in the file system, you might lose all your work. People don't know it.
0:30:35
(Nir)
Usually you don't run into this problem because you're working with IDE and IDE is wrapping it and saving it. Now I want to focus on some hidden features of Git. As I mentioned, if someone is using Git, it's very simple to use Git.
0:30:51
(Nir)
You have a few commands, like this is Ford Model T, my first car, just kidding. For Git, Ford Model T, it's very simple. You sit, you don't have any multimedia, you don't have anything, you simply sit and driving forward. Git is very, very simple.
0:31:08
(Nir)
When I'm teaching it or when I'm looking at Git, this is how I see Git. Git is a very, very complex. You need to know all the details in order to work and enhance your daily daily production. The first thing we're going to talk about
0:31:22
(Nir)
is something called free states. What is free states? In Git, as we know, we're working on a working directory. Once we work on the working directory, we take our code, we're adding it, we're uploading it to the staging area.
0:31:35
(Nir)
It's not a stage, cache, or index. Today, it's referring to the staging area. We take the code, we transfer it to the staging area, and then we commit it to the repository. We commit it to the local repository, and then using Git push,
0:31:48
(Nir)
we upload to the remote repository. The main question is, why do we need the staging area in the first place? Most of all, all other source control, again, I want to be accurate, there is someone who's very,
0:32:02
(Nir)
very similar to Git. In order to make the discussion more simpler, I'm going to say Git and others. But to be correct, there's another one who is very, very close to Git. So just for the simplicity,
0:32:15
(Nir)
it's going to be Git and others. So in other source control, rather than Git, we don't have the staging area. We take the code, we upload it to the server, we're doing a merge, commit, whatever, checking depends on the source control that you're working on and that's it. In Git, we have another layer, the staging area.
0:32:31
(Nir)
So the question is, why do we need it in first place? Since we don't have a lot of time, usually I'm talking about it in my lecture about half a day, seriously, it's very, it's a main concept in Git. Once you understand and you know the abilities and capabilities, you can enhance your work very rapidly and very productivity.
0:32:53
(Nir)
In Git, we have the staging area. The staging area for our discussion today and for our demos is doing something very simple. Each layer, working directory, staging area, and repository, we can look on it as a different file system
0:33:10
(Nir)
which can store different version of the same file. Which means that if in the working directory, I have a file with a text A, I can take this file, I can check in, I can upload between GitHead, I can upload it GitHead, I can upload to the staging area, I can have a different version of the same file.
0:33:29
(Nir)
Then I can commit to the repository and I can have another third version, for example. Usually in real life, we say we put the same file, we upload the same file for the working directory to the repository. But there are some cases where we don't want to do it.
0:33:45
(Nir)
For example, the most common cases, if you work with Git and you see LF and CRLF in the message in the terminal, it means that Git is changing or transferring, converting the file from Windows to Linux and vice versa. And this is the message that you see.
0:34:00
(Nir)
Now we're going to enhance this ability and we're going to do something very, very nice. Let's take the following scenario. I'm working on a project. I have my working directory. In my working directory, I want to set up a env file,
0:34:17
(Nir)
which has values from my local development. For example, localhost, for example, memory size, for example, names, flags for example, memory size, for example, names, flags for debug and so on. I want to have my own local version of the environment file. In the repository, I want to keep the values with the real values,
0:34:38
(Nir)
with the production values that I need. I check out the code, I'm going to a file, I change the values to the one that I want. And once my feature is ready, I want to commit it back to my repository.
0:34:52
(Nir)
Most of the people, most of the users, what they do today, they take the file, they check, be there with the diff tool, either we put it on somewhere else, taking, reverting to the original fine, original fine, cherry pick with only the lines
0:35:09
(Nir)
that you want and so on. It's very, very complicated and it's human error. Git allow us to do it with the built-in capabilities. It's called smudge and clean. And this is something that I want to show you today. In our scenario, we will take a file.
0:35:27
(Nir)
The file will be dot n with any variables. We'll take it. We will change the file. Once we add it to the staging, we'll change the file to contain the production values. And once we check in, we will have the production value.
0:35:45
(Nir)
Once we check out everything in our local folder, we will have the values that we want. While every time that we commit a file, the repository will contain the values that the production needs, which means that we take the file,
0:36:01
(Nir)
we do whatever we want with it, we commit it any way we want, we reduce the file, we do whatever we want with it, we commit it any way we want, we use erroring. Because if someone by mistake change or not by mistake, change value and by mistake committed it to the repository, we will see the bug in the next build
0:36:16
(Nir)
or in two days and so on and so on. So right now, what we're going to do, we're going to turn on the feature in Git and then in Git, listen, whenever you commit the given file, keep my local values,
0:36:34
(Nir)
but set the right values for production. This is the first demo that we're going to see. It's called Smudge and Clean. The mechanism is every time that we're going to see. It's called smudge and clean. The mechanism is every time that we add, we're doing git add, something is happening behind the scenes.
0:36:51
(Nir)
Git can change the content of the file. It's called smudge and clean. We smudge it and we clean it. So every time we take a code, for example, if you're working on a working directory and we add in Git, we do Git add to the staging area, we're doing a clean.
0:37:10
(Nir)
Every time we take the file out from the staging, another filter is taking action and we change the find locally. This is smudging clean. The second thing that I'm going to show you is how can we eliminate or how can we reduce, not fully eliminate, reduce the number of conflicts
0:37:34
(Nir)
that we have when we're merging with Git. Git has a little feature called RERERE. What RERERE does, RERERE is recording its initials for we use recorded resolution. It's simply recording if you have any conflicts and we resolve it.
0:37:52
(Nir)
It's recording the way that we resolved the conflict. And from now on, every time that you will see the same conflict, it will resolve it the way that we want. We have two flags which we can set. First of all, is enabling the feature. And second, we can tell Git listen,
0:38:11
(Nir)
I want you to auto resolve it automatically. Don't even ask me. It's a bit dangerous. That's why it's not recommended, but setting up the flag to enable it every time you have a conflict is fully recommended.
0:38:23
(Nir)
And basically that's all you need to do. Again, we're going to show a demo. We will see it in action. Okay, so it's very simple. The last thing that I want to show, if you have time, you'll see about the time,
0:38:34
(Nir)
it's something called get sparse checkout. What is a sparse checkout? Sparse checkout is if I have a huge monorepo and I have a lot of folders, a lot of content, for example, a common folder, a network, a shared, everything, and now I want to check out only part of the project.
0:38:53
(Nir)
I don't want to do a git clone to all the projects. I just want my part. So using git sparse checkout, I can check out any folder that I wish. For example, in this scenario, if I'm an Android developer,
0:39:07
(Nir)
I can check out my code and the shared folder, the root. If I'm someone else, I can check out again the root, which is shared with everyone and my code. And I can do a lot, a lot of manipulation. This is called sparse check out. Now what we're going to do in the time that we have left,
0:39:25
(Nir)
I'm going to show you demos of all the thing that we just showed. In the slides later on, you have some extra stuff as well. For example, you have some Git log commands which you can see. Again, if I want to search in log, I want to search, for example,
0:39:42
(Nir)
who changed specific line in code, I can do it. If someone deleted search in log, I want to search, for example, who changed specific line in code. I can do it. If someone deleted file in Git, and you want to see who deleted it, again, I cannot do blame, I cannot say it in history because it was deleted.
0:39:55
(Nir)
Using a Git log, Git log dash dash, I can see the last commit, which the file was part of, which means that if it's the last commit, which the file was part of, which means that if it's the last commit which the file was part of, which means this is the commit which deleted the file or the prior to it. So using git log, I can enhance my working
0:40:13
(Nir)
instead of going manually and searching for the history who deleted the file, who changed the lines between, for example, line 10 to 20, I can do it with git logs command. Git log itself has something like 450 flags, the lines between, for example, line 10 to 20, I can do it with git logs command. Git log itself has something like 450 flags,
0:40:29
(Nir)
which is a huge. And again, the combination is different. If I'm putting it in a different order, I will get a different result. So git log is a very, very powerful tool if you know how to use git.
0:40:40
(Nir)
Work tree allow us to work on multiple branches simultaneously. If someone is working with Git, you know that every time you're doing a Git checkout, the content of the folder is changing. If I have a scenario where I want to work on a project and doing a build,
0:40:57
(Nir)
and the build takes something like 20 minutes, I cannot do anything else because my file system is locked. What we do, we simply using WorkTree and we can work simultaneously on multiple branches. I don't have time to show it today, but I'll show you a lab where you can see everything.
0:41:15
(Nir)
In the slides again, you have all my contact information. I'm working in a company called CodeWizard. You know, a company who we do on a daily basis, we're doing DevOps as a service, we're giving DevOps to other companies who need the capability, don't have enough manpower,
0:41:33
(Nir)
don't have enough employees, or don't have enough knowledge to do so. So this is my contacts, you can feel free to contact. In our company website, we are managing something like 100 DevOps communities. So you can join any community that you want.
0:41:49
(Nir)
You can see it in the site itself. So this is for the slides. And now we're going to the live demos. If someone has any questions, feel free to jump. Okay. Jump and ask, and I will try to answer.
0:42:03
(Nir)
The first demo that we're going to show is what I spoke about. Again, you will get an email with all the links. I have in my GitHub a repository called GitLabs, with a lot of demos. Each folder contains a running demo, it's showing everything.
0:42:21
(Nir)
Today we're going to focus on three of them, which is very, very short because we don't have a lot of time. The first thing that we're going to focus on three of them, which is very, very short, because we don't have a lot of time. The first thing that we're going to do, we're going to talk about smudge and clean. Smudge and clean are the mechanism
0:42:34
(Nir)
that whenever we check file to Git or check it out, we can change the content. So let's look at the code. Okay, let's look on the code. Okay, let's look on the code. What does the demo does? And again, if you have any question,
0:42:50
(Nir)
feel free to jump in and to ask. Okay, what we're doing is very, very simple. We have a local variables for local IP and production IP. We set an environment file. The initial value is this one. This is the initial value of the end file.
0:43:09
(Nir)
Now in the demo, what we're going to do, we're going to change the value locally to something that we want. We're going to commit the file to Git. We will see that it will have a different value in the Git repository.
0:43:26
(Nir)
And once we check out the file, we will get the desired value that we want locally in our system. Someone asked a question, why using a CLI? In CLI, you can do a lot of things that you cannot do with plugins. For example, as I mentioned, Git log, for example, has something like 450 flags.
0:43:47
(Nir)
There isn't any developer who can develop GUI for such a thing because you have a lot of permutations as well. The IDE supports and give you something like 80 percent of the thing that you need. But if you need something is edgy,
0:44:03
(Nir)
if you need something is edgy, okay? If you need something is edgy, so you need to do it manually. For example, I don't have any IDE, okay? That I'm answering the question for the chat. They don't have, sorry, I didn't mention it. I'm answering the question in the chat.
0:44:19
(Nir)
Sorry, that's why I did a shift. I didn't mention it. So if you want to do something a bit different that the ID support, you cannot do it. For example, you cannot set it or smudge and clean. You cannot set it through the IDs.
0:44:34
(Nir)
So in our example, what you're going to do, we're going to change the file. We're going to commit the file to Git. And after we commit, we'll see the value. The content, the secret basically behind this thing is very simple. Again, you have read me in the lab displaying everything. What we do, we're doing the following things.
0:44:54
(Nir)
We're setting up filters. As I mentioned, smudging in our filters. We're setting up a filter for the clean and the smudge. We don't have to set both of them. We can set any of them if we want, and we can set none. Once we set those filter, those filter take action
0:45:14
(Nir)
and depends on the command that we do, for example, if we do in git add, once we do a git add, so the clean is taking in action, what the command or the suite that will be written here will be executed. And once we check out, this smudge is executed.
0:45:33
(Nir)
So in our examples, as I showed the code, let's see it running in running example, and we will see everything. Okay, so the first thing locally, we are building the .env file. This is what you have locally in our system.
0:45:49
(Nir)
Again, this script is the one that I just show you. It's in the GitHub repo. Everything is working over there and you can take it and use it. We're setting a .env file. This is the initial state.
0:46:01
(Nir)
What we're doing now, we're adding file. We're committing. I did a git add to the file. Once I did, you have here expansion of what we did. I did a git add to the file. Once I added the file, I did the command of git add,
0:46:17
(Nir)
git automatically changed the value. How did it do this? Or what was the good thing? What was the way that it happened? This is the secret lines 83 and 87. I set up local filter.
0:46:35
(Nir)
For the clean and smudge and the clean is basically searching for the purpose of the demo, searching the line with the database IP and change it every time I commit searching for the purpose of the demo, searching the line with the database IP, and change it every time I commit it or editing to the production.
0:46:50
(Nir)
Every time I'm checking out the file, is checking it out with the value that I want, which means that every time I'm adding file to Git, I'm making sure that no one will commit something that I don't want. Every time it's checking out,
0:47:09
(Nir)
it's doing whatever I want to do. It's a very, very powerful tool. As you will see in the readme file, I mentioned that we can use, we can have it with a lot of extension of Git. They're using it, like Git Encrypt.
0:47:29
(Nir)
Okay, Git Encrypt, which allow us, for example, to encrypt the content, Git LFS, which allow us to work with large file storage, or large file, binary file, which is not good to place in Git, but you don't have to try to take to explain why,
0:47:46
(Nir)
both of them are using the smudge and clean. A lot of people are using those tools. I'm not aware how does Git is using them behind the system. So the way that Git is doing it is using the smudge and clean, sorry, I'm not in the right place,
0:48:02
(Nir)
the smudge and clean filters., I'm not in the right place, the smudge and clean filters. Okay, I mentioned it here, git clear, git lfs and so on. So let's continue with the demo. So once we committed the file, this is the value, which will be in the git repository. Okay, this is you see the content in the repository.
0:48:21
(Nir)
Again, in my local folder, I had a different value. This is my local folder. This is the content in Git. Now I'm showing the content of Git. Now what I did, I removed the older file. I check out the file from the Git repository.
0:48:41
(Nir)
And once I check it out from the Git repository, I didn't get the value. This is the value committed in Git. I didn't get out the value which was committed. I get the value that I wanted. Okay?
0:48:54
(Nir)
I got the value that I wanted from it. For example, as you see here, this is the content which committed to Git. And once I check out, this is the content which I get locally. It can save you a lot of bugs.
0:49:07
(Nir)
It can save you a lot of frustrations. Someone mistakenly changed the file. Someone did something wrong. This is what we called local, in this example, local changes, but I'm using smudge and clean to do it.
0:49:25
(Nir)
The second demo that I want to show you it's something called RERE. What is RERE? Every time that we're doing a merge, we usually with rebase sign some cases, every time we do the merge, Git try to merge content.
0:49:39
(Nir)
In some cases, we have conflicts. Conflict are very, very frustrating and it's hard to handle them. Rarara is a hidden feature in Git, which allows to tell Git, listen, every time that you see a conflict,
0:49:53
(Nir)
I will resolve it the way that I want. And from now on, I want you to suggest to me what is the right way to resolve it. So let's see it in action. What we have, what we have here,
0:50:08
(Nir)
let's go over the code first, sorry. This is lab 32. Okay, what we're going to do, and let's go over the code, close this one so we will have ability to see. Okay, what we're doing in the code,
0:50:21
(Nir)
we initializing a Git repository, we generating, I'm generating few branches. I'm adding the same line to all the branches, which will cause a conflict. For example, in each branch I'm putting, I'm adding, you see echo,
0:50:39
(Nir)
I'm adding the file, the branch name and the content to the same line, for example, like here, a echo line from, this is a branch. Let's go over it very, very straight. First of all, we're creating the repository.
0:50:54
(Nir)
Then we just initializing git repository. We're creating, once we have the main branch, we're creating another branch, the second branch. On the main branch, okay? On the main branch, okay, on the main branch, what we're doing, we're adding line 39,
0:51:09
(Nir)
we're adding line from branch, in our case, we're main, to the file. We continue and we're committing the changes on the branch main. On the second branch, I'm doing the same thing. I'm going to the demo branch
0:51:23
(Nir)
and then I'm committing the same line, but with a different branch name, which means I will have a conflict. Once I have a conflict, I'm going to merge those two branches. Again, since I have a conflict,
0:51:36
(Nir)
the merge will cause a conflict and let's see how does Git resolve it. So I enabled the flag of the error array. Once I'm enabling it, Git will resolve the config the way that I want. In my case for the demo,
0:51:53
(Nir)
I resolved it in lines 120 to 122. I resolved it with a text resolution. You can put anything you want, obviously. This is the way that I decided to resolve it. The next time, what I'm doing, I'm reverting the code back to the previous version
0:52:12
(Nir)
because I want to do a merge again and get the conflict back again. But this time, Git will auto-resolve the conflict for me. This is the RER the error, which again, which can lower the way that we merge manually. Every time you merge manually, we can do mistakes, human mistakes, human error.
0:52:34
(Nir)
This is reducing the error that we can get once we do it. So let's see it in demo again. The script, once you see we committed to the main, this is line for main, we committed to the demo branch, this is line for demo branch. Now we're doing the merge.
0:52:49
(Nir)
Once we do the merge, we have the conflict. This is the result of the conflict. As you can see, since I have enabled the flag, it's telling me, listen, I'm now recording the way that you will resolve the conflict. So this is the conflict.
0:53:08
(Nir)
Now I resolved it with the text, name resolved. This is the way that I decided to resolve it by nothing. That doesn't care how do you resolve it. And now I'm resolving and committing this change to Git. Later on, I'm checking out, you see, I'm checking out the content.
0:53:29
(Nir)
This is, I checked out again, the older version that I have, and I did a merge again. Now, when I did the merge, it's telling me, listen, I'm using the previous resolution that you gave me.
0:53:43
(Nir)
You see here, it's recorded the resolution, and now it's using the resolution that you gave me. You see here it's recorded the resolution and now it's using the resolution that they gave me. And this is the content of the resolution right now, which means that now every time that I have a conflict, I don't have to struggle it with again. We can use this for a lot of other things.
0:54:03
(Nir)
After them I will show some few tips, and that's it. Any questions so far, anyone? Good. Let's go to our last demo, and then I'll show you a few more tips, and then we will summarize. The last demo is lab number 12, in this case.
0:54:23
(Nir)
What I'm going to do to show you now, I'm going to show you how can I check out only part of repositories. I don't want to check out my all repositories since it's a huge one and takes a lot of space and a lot of time, I don't need it.
0:54:39
(Nir)
Whenever I'm doing a build, I'm using a build server, CICD, I don't need all the repository, I just need to build something very, very simple. So how do I do it? This demo, we create a folder, two repositories, this is a sparse demo, this is the checkout demo, the sparse is the full repository,
0:54:58
(Nir)
and the sparse checkout is the small repository that we check out out of the main repository. What do we do here? We create three different files, each one in different folder. We commit it to Git on the folders,
0:55:11
(Nir)
and we will see the content of the folders. Now, I do a walk with the checkout. I'm doing a Git clone. I'm setting the flag. Now, I'm doing a Git clone. I'm giving it, I have a lot of options that I can do.
0:55:29
(Nir)
I give it a lot of options that I can set off. This is line 48, okay? I'm telling it, listen, I want you to use checkout with sparse checkout. I did a git clone with a filter branch and sparse, I added values.
0:55:47
(Nir)
I'm checking out the product without first checking out the files, only marking it that later on we check out some files. So I cloned with the spark, then I'm setting up the spark in it. And then I'm setting which folders I want to check out.
0:56:06
(Nir)
I can have a list of folders, I can have a common network, anything that I want. And once I'm doing SkitsPath checkout with the folder, it will only check out the desired folder that I want. Let's see the demo. I have a lot of demoes.
0:56:25
(Nir)
This is the first one. Okay. As you can see, I committed code to the repository, three files in three different folders. Now I'm going to check out only the server folder.
0:56:39
(Nir)
So I did a clone with the sparse. Okay. I'm cloning it to the, like a git clone to the sparse. Okay, I'm cloning it to the target clone to the folder. And since I set up only the server folder, it checked out only the server folder for me.
0:56:56
(Nir)
This is the last step. Last step that I want to show you, I didn't demonstrate it due to lack of time, but I will show you in the repository. We have the smudge and clean using the smudge clean. We have two or multiple other demos, for example.
0:57:11
(Nir)
Let's say that now your build, every time we do a build, your version file is updating. The version is bumping up. Every time you will do a merge, you have a conflict because locally and on the remote,
0:57:23
(Nir)
the version is different. I can use smudging clean, the same mechanism to tell Git, listen, every time now that you want, every time that I'm changing file, I want you to use my version. This is, for example, in this demo.
0:57:41
(Nir)
I want to use my version instead of having conflict or having something using smudging clean, I'm telling it, listen, every time you see something different in the file, I don't care what's in the version text,
0:57:54
(Nir)
I don't care what's in the image, in the logo, whatever, whatever you want, use my file.
0:58:00
(Speaker 6)
That's it.
0:58:03
(Nir)
Any questions?
0:58:05
(Speaker 8)
Cool. That's it. Any questions?
0:58:09
(Nir)
Cool. As I mentioned, you will get everything. It's in my GitLab Git repositories. You will have a list of the labs. There are a lot of other labs as well, which we did not cover. That's it for me.
0:58:20
(Ganesh)
Awesome. Thanks, Nir. I was actually involved in a migration from SVN to Git a few years ago. And I wish I'd have known about CodeWizard then, because we had some serious problems. I'm sure you would have been able to help us.
0:58:36
(Nir)
That's what you're doing.
0:58:37
(Speaker 7)
That's what you're doing.
0:58:38
(Speaker 6)
That's how we daily walk.
0:58:39
(Ganesh)
Yeah. Yeah. A lot of integrations. I'm not a developer, but I know we had serious issues with SVN. So super cool. Thanks for the demos. Really appreciate it. Before we wrap up, I want to leave you with some thoughts.
0:58:52
(Ganesh)
A lot of startups were hitting this series A moment where everything speeds up, your team size, your code base, your customer demands, but your infrastructure, it doesn't always scale the same and hiring the right people can take some time. That's where GlobalDots come in. We don't replace your team, we accelerate it. We embed senior engineers, we work alongside you.
0:59:13
(Ganesh)
We unblock bottlenecks. We introduce you to smart people like Nia and Oded and we can help you get your roadmap moving faster. So if you're feeling the pressure of scaling, but you want to stay in control, maybe this is a model worth exploring.
0:59:26
(Ganesh)
To help you with this, we're offering a free 30-minute session with one of our senior engineers. No strings attached, just for a second opinion on how your current setup is holding up or where friction might be hiding. We have a link there in the chat,
0:59:40
(Ganesh)
and also we'll put it on the post-notes recordings. And we'd love to keep you scaling without breaking. In closing, thanks again. This is a one of a kind special Cloud Next and podcast episodes. Super thanks to our speakers, Oded and Nir
0:59:56
(Ganesh)
for sharing their great experience and being so generous with their wisdom today. And any closing remarks from you guys?
1:00:15
(Oded)
So as I mentioned at the beginning it was a great opportunity for me to go back in time and revisit the decisions we made and think about the lessons we learned afterwards. Hopefully it was helpful. Of course I will share the materials just like Nir and thanks everyone for the time today. And thanks to Nir.
1:00:33
(Nir)
Nothing else. I think I gave some inspiration to other people as well. As I mentioned, I'm developing and working on a lot of open source and good luck to everyone if you need any help anything.
1:00:47
(Speaker 5)
Having a lot of communities which helping others.
1:00:52
(Speaker 4)
Pleasure and thanks for the war story Nir, I appreciate that a lot. Pleasure and thanks for the war story Nir, I appreciate that a lot.
1:00:58
(Ganesh)
Don't forget you can find this episode and more like it on Spotify and global.youtube channel. Thanks very much and see you again next time.
