An Internship on Ramp’s Forward Deployed Engineering Team

Starting My FDE Internship

My internship on Ramp’s Forward Deployed Engineering team (FDE) has been a great learning experience for me as an engineer and aspiring entrepreneur. FDE at Ramp was created to build product features that solve the problems of Ramp’s largest enterprise customers. Learn more about FDE here.

In my first week, I was immediately assigned to help address an enterprise customer’s concern, unlocking the potential to add millions of dollars per month in additional Bill Pay spend. FDE requires the technical flexibility to work across the entire Ramp codebase, the speed to ship good products before customer deadlines, and the business sense to understand customer problems in order to design the most efficient solutions. The art of scoping a customer request from a vague ask to an actionable feature request through understanding pain points was one of the most impressive skills I saw my teammates demonstrate right from the start of my internship!

After shipping my first code on day 3, I was introduced to my full internship project: automatic hotel receipt retrieval.

Automatic Hotel Receipt Retrieval Project

Each year, Ramp’s enterprise customers alone spend tens of millions of dollars on hotel stays for their employees. Retrieving “folios” (hotel receipts) for these transactions generally requires an employee to request a copy after their stay. We understand that the last thing a busy employee wants to do after a long business trip is to manually acquire and upload a receipt onto their expense management system. As a result, many customers do not strictly enforce a folio requirement for hotel transactions and many folios are never uploaded to Ramp. Since folios are the official source of tax information for these transactions, companies miss out on millions of dollars of tax refunds from Canada, the EU, and the UK every year (VAT-systems offer significant tax rebates for employee lodging).

My project was to investigate and implement solutions for retrieving hotel folios automatically without employee involvement after stays. I started with an investigation into project value-add, identifying that we could save over $400,000 yearly in tax rebates for a single enterprise customer and hundreds of work hours for the company’s finance team. This extrapolates to tens of millions of dollars of potential savings across Ramp’s entire customer base and tens of thousands of hours saved. With that in mind, I started work on the first sub-problem of matching employee hotel bookings with Ramp card transactions.

Matching Hotel Bookings with Ramp Transactions

Large enterprises almost always employ a “travel management company” (TMC) to handle travel bookings for their employees. Company employees log in to their TMC platform to book flights, hotels, and car rentals using their Ramp card.

Prior to my internship, FDE had developed a framework for integrating with these TMCs, automatically fetching booking data to enrich card transaction data. The TMC integrations complete this work in the following steps:

1. Ingest TMC booking data via polling their REST APIs or receiving it via a webhook.
2. Process raw booking data into standardized records in a Ramp schema.
3. Match booking data with corresponding Ramp card transactions. This requires doing fuzzy bipartite matching between bookings and transactions to obtain as many correct 1:1 matches between bookings and transactions as possible.
4. Tag transactions with information obtained from the booking data, including memos, accounting data, users, and more.

Basic diagram of Ramp TMC integration pipeline.

One major problem with the existing TMC integrations pipeline was that the match rate for hotel bookings to corresponding Ramp transactions was very low at only approximately 20% relative to the 90%+ match rate for flights. My project was to improve the matching procedure for hotels to increase this match rate as much as possible. Why was there such a discrepancy between flight and hotel match rates? Flight data generally exhibits consistent prices and similar times between bookings and transactions, whereas prices, payment dates, hotel names, and even user/card identification information varies considerably between hotel bookings and transactions. As a result, fuzzy matching on multiple features is necessary to achieve a high match rate. Additionally, since not all hotel bookings made on TMCs are ultimately paid with Ramp cards and not all hotel-related card transactions are related to TMC bookings, it is very challenging to quantify the total accuracy of any matching algorithm. In other words, we cannot accurately generate a correct “confusion matrix” for results.

Matching Ramp card transactions with booking data from TMCs is a tricky unsupervised bipartite matching problem with no way to correctly assess the total quantity of correct matches or the true quality of any specific match.

I spent several weeks of my internship primarily thinking about this problem, in addition to managing incoming travel-related requests from one of Ramp’s largest customers. I experimented with a wide variety of solutions to this hotel booking/transaction matching problem, including heuristic-based and pure ML approaches. My final solution reached an over 80% true positive match rate across a variety of enterprise booking practices, while minimizing incorrect matches using very conservative match thresholding.

My ultimate solution utilized the Gale-Shapley stable matching algorithm with partial preference lists, a variety of important pre- and post-matching filters for ensuring that only very high confidence matches are made, and a tuned linear model for weighting relevant features to compute scores.

The Gale–Shapley stable matching algorithm pairs elements in two sets, A and B, one-to-one. Each item in both sets has a “preference list” or an ordering of items in the opposite set. Items in A propose to items in B in order of preference, while items in B accept the best offer they’ve seen so far. Critically, items in B can later drop an existing match for a better match. This process continues until all objects are matched and guarantees a “stable” outcome where no two objects would rather be with each other than their assigned partners.

In our use case, sets A and B are the set of bookings and the set of transactions, which are different sizes. Furthermore, each item’s preference list only consists of a subset of items in the other set that pass pre-filters (this is a slight modification to the original algorithm). Preferences are ordered based on match scores, which are computed using a trained linear model.

To train the underlying feature weighting linear model, I utilized derivative-free optimization methods with an objective based on the following components:

1. Maximize total matches – the more matches we make, the more value the integration system adds to the customer.
2. Minimize a hinge loss penalty – each element should clearly prefer its chosen match over the second best match. This serves as a proxy for match confidence and correctness.
3. Minimize an additional “candidate penalty” – tune a learnable score threshold to separate matches and disincentivize matches from having multiple possible candidates above this threshold. This score threshold can be adjusted to trade-off match quality vs. quantity.

While it took me several attempts to achieve this final algorithm, the resulting solution is stable, effective, and has been deployed into production for some of Ramp’s largest customers.

Implementing Automatic Receipt Retrieval

After completing work on hotel matching, I spent the remainder of my internship focused on actually fetching receipts from hotels automatically. While scoping, I was able to determine that over a third of all Ramp hotel-related receipts and closer to 40% for enterprise customers come from only a few large hotel chains. These chains offer forms on their websites that allow for requesting hotel folios directly. For other hotel receipts, however, fetching receipts can only be done via emails and, most often, direct phone calls. Given this data, I began exploring how to use browser agents to submit web forms and voice agents to call hotels on behalf of customers in order to retrieve receipts automatically.

I was able to quickly build and ship prototypes for fetching receipts, relying on new infrastructure work developed by our Employee Experience and Applied AI teams. I found this work to be a very different flavor than my past work on hotel matching, emphasizing the variety of exciting projects that Ramp engineers regularly undertake. In my case, I utilized Playwright and Stagehand, an LLM-powered browser agent library, both of which enable powerful browser automations. While most static forms can be filled out automatically with hard-coded xpaths and other HTML selectors via Playwright, Selenium, or other tools, LLM-driven semantic selection and action on form pages can be useful in ensuring stability in the event of target form or website updates. We generally advocate the approach of building static form automation scripts backed up by LLM-powered tools in the event of errors. This provides the fastest, cheapest, and most stable approach overall. Browser agents are already being deployed at Ramp and for some of our enterprise customers, with plans for larger roll-out on the immediate horizon.

The underlying Ramp internal voice agent infrastructure is based on Pipecat, an open source framework for building voice agents, and integrates Twilio, Deepgram speech-to-text, OpenAI GPT for response text generation, and ElevenLabs text-to-speech, over websocket. This results in an agent that speaks in real-time, sounds like a human, and is able to effectively communicate with humans over the phone to request that hotel receipts be sent to their owners. All voice AI agents communicate their identity as an AI at the start of each conversation and receipts are only ever sent to their owner’s email address. Voice agents are still experimental but the hotel receipt retrieval use case is an excellent demonstration of the far-reaching possibilities for deploying this technology.

After a receipt lands in an inbox, Ramp’s existing mail inbox integrations are able to effectively capture it and pair it with the correct corresponding card transaction. Watching my code in action, fetching and retrieving receipts for real hotel stays in the browser was really fun, as was practicing folio retrieval with our Ramp voice agents on the phone!

Key Learnings

My biggest takeaways from my Ramp internship correspond to some of the FDE team principles outlined in our team blog post:

1. Always be Scoping – During my Ramp internship, one of the most impactful skills I learned was how to scope problems more effectively, enabling me to impact and solve business problems using my technical skills. As an aspiring entrepreneur, this is a critical skill for me to learn and my desire to learn how to work on solving problems directly for customers brought me to FDE in the first place.
2. Do More – Velocity and speed of incremental progress is very important at Ramp and by the end of my internship I learned to “ship” code faster and in smaller chunks to speed up my overall work output. My natural inclination is to build components of projects to total perfection and to ship large amounts of code in bulk, but I learned that this mindset ultimately results in slower progress. A better mindset and the Ramp one is to ship good work as fast as possible, gather feedback from customers for iteration, and ship incrementally such that progress compounds as opposed to experiencing only linear growth. I expect to take this mindset with me in my career and potentially as an entrepreneur one day.

Apart from these lessons & takeaways, I also learned the importance of a great team! Ramp’s FDE team is a group of incredibly talented, but also fun and compassionate individuals with whom I was lucky to work with this summer. It is really amazing what such a team can accomplish when working together toward shared goals!

Reproducing the Ramp logo at our team offsite in Sonoma!

Reflection

As I approached the end part of my internship, Ramp & FDE had one last surprise. In my 10th week of my internship, I had the wonderful opportunity to join FDE for an offsite, glamping in Sonoma, California! I had not visited the San Francisco area for over a decade and really enjoyed the trip, getting to know everyone on the team even better, playing way too much cornhole, watching comets in the sky, exploring the Sonoma area, and even riding a Waymo for the first time in SF.

My Ramp internship was an excellent learning experience for me as I continue to think about my career moving forward. I highly recommend working at Ramp to other software engineer interns who are looking for an opportunity to join a fast-paced, rapidly growing team with a strong engineering culture. A special shoutout to my manager Lachlan for making me feel welcome and for supporting me throughout the entire course of the summer!