Descriptify

My biggest mistake as an RPA developer

Mar 2022

I was an RPA developer for 4 years - from 2017 to 2021. In late 2019, I made one crucial decision that led to my being chosen as one of UiPath's 62 MVPs for the year 2021. This decision was to stop keeping up with the latest RPA trends and focus instead on mastering traditional software development.

However, this wasn't always the case. In my first two years as an RPA developer, I stuck to using only native RPA workflows for automating processes. My background was not in IT, so I assumed that traditional programming and scripting were not for me. I viewed RPA as a different programming paradigm from text-based programming and refused to stray from it. I was smitten by how powerful RPA was and how quickly I could learn it. This also blinded me to its limitations.

The size of the RPA market increases by orders of magnitude every year. RPA platforms today are far more powerful than they were back in 2017. As an RPA developer, this makes it tempting to 'specialize' in RPA and not stray outside its boundaries. I now outline 3 reasons why thinking that way is a mistake.

  1. GUI based automation is always a compromise

    2. Automating any software process, at its essence, involves moving data from one place to another using a series of commands. Over the decades, programmers have perfected the means to do this using the aptly named command-line. However, to extend a computer's use to the everyday user, we created a more intuitive user interface - with a mouse pointer, buttons, text fields, touch-screens and so on.

    Enter the GUI - the Graphical User Interface. We are now so used to GUIs that we take them for granted. Yet, as programmers, we need to remind ourselves that every time we use a GUI, our computers perform digital gymnastics to accommodate our needs. Each time you click a button on your screen, the computer invests tremendous effort to translate this gesture into a command. It has to

    • Render the button correctly on the screen
    • Track the position of the mouse pointer
    • Register the button's click
    • Execute the corresponding command

    Alternatively, issuing a command using a line of programming code is like speaking to the computer in its native language. It is efficient and robust with minimal information loss.

    Most native RPA automation is GUI based. But let us take a moment so that this sinks in. GUI based automation involves instructing a bot to communicate with another program via the UI. This is analogous to forcing two native speakers to communicate via charades. GUI based automation is always a compromise because there is invariably a more efficient way to perform the same task under the hood. This brings me to my second reason.

  2. Native RPA processes are fragile in production

    3. For the longest time, I failed to realize how fragile GUI based automation was. Since I automated every process on my own computer, I was unable to anticipate how running it in production would be different. However, when I moved to a team that maintains RPA processes in production, everything changed. More than 80% of my team's time went into fixing broken processes, leaving us with little time for automating new ones. This fragility mainly stems from how production environments are invariably different from the laptops and the test systems on which we develop our automation. An automated process of a 100 sequential steps is only as strong as its weakest step. Therefore, it only takes the smallest change in the GUI to derail an entire process.

    As an RPA developer, you have often experienced this yourself. I am yet to see an RPA process that has executed perfectly in production the first time. However, one might be led to believe that this is changing. With advancement in RPA technology such as smart selectors and computer vision, RPA processes can now be more resilient to changes in the GUI. This leads me to my third reason.

  3. Commoditization

    4. To automate a web-application in the pre-RPA era, you needed to inspect the web page on the browser and sift through the complex mess of HTML and CSS to find a reliable selector. With RPA, a developer merely has to click on the element to retrieve its selector. In the next iteration, we had intelligent selectors that supported regex and included some fuzzy logic. The next leap forward was computer vision - RPA software can now look at a screen and identify text fields, radio buttons and check-boxes much like a human user does.

    RPA platforms today use a combination of these approaches to identify selectors. But one thing is clear - the smarter the RPA platform, the dumber the developer needed to work it. In the span of 4 years, the skill required to automate a web interface has gone from a solid understanding of HTML and CSS to merely pointing to a region of the screen.

    With the rapid maturity of RPA platforms comes the commoditization of RPA jobs. The easier it is to automate with an RPA platform, the lower is the skill level of the employee needed to work with it. This trend is analogous to industrialization in the restaurant business. In the past, you needed to employ a chef to run any restaurant. Yet, industrialization makes it possible for you to run a fast-food joint with a middle-school dropout. Yes, chefs remain valuable in the world today, but only because they sharpen their craft and constantly reinvent themselves.

***

Early in my RPA career, my mistake was not to have recognized these factors. I will now outline 3 measures I wish I had adopted sooner.

  1. Think like a software engineer

    2. In December 2019, I read Bob C. Martin's Clean Code, which transformed the way I looked at computer programming. Reading the book allowed me to examine a piece of code from a seasoned software developer's perspective. I noticed how one needs to not just solve the immediate problem at hand, but also design for the second- and third-order problems that can crop up downstream. I also realized that at its core, RPA development is not too different from good old software development. Besides, the craft of software engineering, being decades older than RPA, has solved most problems that RPA developers are only becoming aware of.

    Apart from reading books such as Clean Code and Learn Python 3 the Hard Way, it also helps to talk to software engineers how they automate stuff. I would often explain a scenario I was automating to my developer friend and ask him how he would go about it. This helped me discover wonderful alternatives to GUI based automation, which leads me to my second recommendation.

  2. Learn to automate using traditional programming languages

    3. All RPA platforms are built on top of a traditional programming frameworks. Most RPA automation is done on .NET platforms, so under the hood, they employ C# or Visual Basic. Start with simple workflows and try to bypass the RPA platform by using the underlying .NET language. This way, you obtain a much deeper understanding of how RPA software works. Besides, you also realize that in several situations, a couple of lines of code can achieve the same end result as a convoluted RPA workflow that spans two lengths of your monitor.

    Another way to gain more experience here is to create custom activities for the RPA platform that you most often use. This experience gives you invaluable insight into its inner workings.

    A fantastic resource I would recommend to learn to automate with Python is Automate the Boring Stuff with Python.

  3. Integrate RPA with traditional software

    4. Once you start thinking like a software engineer and add some programming chops to your toolkit, you can craft elegant processes that combine the centralized orchestration of an RPA platform with the robustness and efficiency of traditional programming. With .NET, you can automate tasks on most Windows applications. You can map network drives, integrate DLLs, create custom activities and all of this will turbo-charge your RPA code. For integration with SAP, explore Stefan Schnell's SAP Scripting Tracker and its magical workings. You could automate RPA deployments by building a CI-CD pipeline. This last integration scenario was the one I had most fun exploring, and went on to become my most valuable contribution to the RPA community.

    ***

    Some of what I have written above might be hard to stomach. RPA vendors and industry insiders are unlikely to offer this perspective, and I myself was able to see it clearly only after moving on from an RPA career and looking back at the industry with a fresh pair of eyes.

    Also, my advice to go back to traditional software development rather than stay up-to-date with the latest RPA trends might come across as old-fashioned. However, if you intend to take a long-term view of your career as a developer, you need to master old-fashioned constructs of your craft that have stood the test of time.

    I only wish I had realized this sooner.

    PS: This article made it to the front page of Hacker News. I found the discussion there to be representative of how software developers perceive RPA. It also yielded a handful of useful insights.