Meet our customers at our 2024 Customer Conference
Upwork

Watch the video to see how Upwork leverages Promoted.ai for unified search and native ads optimization.

Watch Full Video Interview

Sree Kamireddy
Sree Kamireddy

Senior Director of Product, Next Gen Search

Upwork Interview Transcript

Intro - 0:00 My name is Sree Kamireddy. I lead the search and recommendations team at Upwork.

Launch Impact - 0:05 We are able to see positive improvements in contract starts. We also shipped a recent A/B test to deliver ads on newer surfaces and both of those we were able to do at a much quicker pace given Promoted is built from the bottom up to be able to deliver both ads and organic search results.

We all work together! - 0:52 So it’s very interesting, it’s very complex, and it’s high scale, and Promoted does this for you anyway, right?We all work together! Absolutely!

Using the Best - 1:01 Upwork takes a lot of attention to using the best out there to build that perfect match engine for the clients and talent.

How You Know It Works - 1:15And, how do you know that search, discovery, and your native ads are working? This is a common set of metrics that we clearly define —our success metrics. And, we look at a lot of variables. And in terms of the metrics, all the way from —are the contract starts increasing —that’s just one metric, and it can be a very short-term metric, so we also look at, are they hiring managers coming back again to do the hiring? Are the hiring managers coming the third time? And are those factors increasing? So not only contract starts are increasing, the retention is increasing. And, we’ll hopefully start doing more of also optimizing directly to increase the gross services volume in itself.

Measuring Unified Ad Impact - 2:03 And ads is another dimension, which is: we don’t want to, obviously, just get ad revenue at the cost of experience and the cost of the organic ranking. So that has its own set of metrics. So it’s a complex slew of metrics to make sure that we are delivering the best possible experience for the users at any point in time.

Promoted Increases All These Metrics - 2:26And is Promoted optimizing and increasing all of these metrics?

Multi-Objective Optimization - 2:29 I think that’s one of the good things about Promoted: the objective functions we can definitely work on figuring out. We can do our internal analysis to see, hey, which objective function should we optimize for. And Promoted, the way the stack is set up, enables different combinations of objective functions. We started with some click optimizations, we did hire optimizations, and all of that, and easily combined them. There are blender functions and all that —not to get too geeky, but all of this is definitely all possible.

Continuous Monitoring of Progress - 3:05 The way I find value is that there’s definitely a Slack channel where I constantly see new models being shipped and what are the different scores for the new models so that you can compare on a day-to-day basis. You can compare as this new model with maybe newer feature inputs —but maybe not— most of the time, it’s newer data. And that newer data is causing this newer model to be retrained and potentially be a little bit better on a day-to-day basis. That part is super valuable. Super valuable as a collaboration. Also, personally, it’s super valuable because I can kind of see the improvements happen. So that’s the offline experimentation part for sure.

Multiple A/B Testing - 3:52 On the A/B testing, we do definitely do A/B our test while you [Promoted] also A/B test within that A/B test. So if we launched Promoted to treatment, you [Promoted] are able to A/B test within that treatment to see if some of the models [are better] and you can move faster within that A/B test, I think that is great.

Continuous A/B Testing - 4:15 But we [Upwork] continue to do A/B test[ing] because that allows us to really validate whether we are doing the right things for the customers.

Recommending Promoted - 4:30 I would recommend Promoted because one, the team is really passionate about this problem space. Like, I definitely see in everyone that they care about the search and marketplaces and doing this matching process really effectively. The team is clearly, very experienced and skilled in this. Everybody has done this in their past life. The speed of iteration, I think is very, very important. And, Promoted has the infrastructure and the stack to go fast and iterate fast and build newer models and ship quickly and that allows for innovation.

The Best Possible Thing - 5:07 This combination of being passionate and having the skills and being able to innovate: that’s a deadly combination. It’s like a top notch professional team. Like, I don’t have second thoughts like, when we are collaborating. And, like, okay, your team is going to do the best possible thing at that point in time.

Promoted Really Gets It - 5:30 I feel like I can talk about some geeky stuff about search and recommendations and this whole match. And I feel like your team really gets it, and that’s pretty exciting.

Promoted Infra to Move Fast - 5:45 Promoted has the infrastructure to move fast. So essentially, all the goals that we were trying to trying to hit, using their infrastructure to move fast, we are able to hit them sooner and learn what works and what doesn’t work sooner.

Sree’s Deep Expertise in Search AI - 6:01 I started in the space long before machine learning was the cool deal. I was a research assistant in this lab called Group Lens. They created a lot of recommendation systems back in the day. I was at Microsoft after that, was part of the founding team of Bing. Then there’s been big companies and large companies primarily using data and AI to drive business outcomes.

The Joy of Solving Search and Ads - 6:26 It’s a complex problem, it’s a complex problem, has complex solutions. Changing all the time. Yeah, it’s a puzzle, it’s a puzzle that you have to solve for and that’s exciting! Interviewer: Solving employment worldwide. Yeah! Think think about it as a game you’re playing on a day to day basis at work. And like how cool is that. Like you come into work, you kind of solve a puzzle and make the clients and freelancers happy. You solve another puzzle another day. It’s very, very interesting. Not a single boring day.

“We are able to see positive improvements in contract starts. We also shipped a recent A/B test to see if you can deliver ads on newer surfaces and both of those we were able to do in a much quicker pace given Promoted is built from the bottom up to be able to deliver both ads and organic search results.”

“This combination of being passionate and having the skills and being able to innovate: that’s a deadly combination. Promoted is a top-notch professional team. I don’t have second thoughts when we are collaborating. Your team is going to do the best possible thing at that point in time.”

Sree Kamireddy
Sree Kamireddy

Senior Director of Product, Next Gen Search

In the News

Upwork Launches Unified Search and Ads Optimization Using Promoted.ai

San Francisco, CA, August 13, 2024 — Promoted.ai is excited to announce the integration of its powerful unified search and native ads engine with Upwork (NASDAQ: UPWK), the world’s work marketplace that connects businesses with independent talent. This collaboration has led to the launch of Upwork’s unified search and native ads optimization. This initiative has been a collaborative effort between Promoted.ai and Upwork’s strong search, ads, and AI engineering teams.

About Upwork

Upwork (NASDAQ: UPWK), the world’s work marketplace and AI-powered platform, connects businesses with independent talent from across the globe.

In a recent development, Upwork launched unified search and native ads optimization powered by the Promoted.ai platform, working closely with Upwork’s strong search, ads, and AI engineering teams. This advancement aims to increase overall hires, contract starts, and ads revenue by enhancing Upwork’s freelancer search and ads capabilities.

Approximately 85% of Upwork’s revenue comes from its Talent Marketplace, driven by marketplace search and discovery. In Q1 2024, the company’s revenue grew 15% year-over-year to $190.9 million. Promoted’s marketplace search and ads engine now powers Upwork’s freelancer search and ads, further strengthening its market position.

Serving everyone from one-person startups to large Fortune 100 enterprises, Upwork provides a powerful, trust-driven platform that enables companies and talent to work together in innovative ways, unlocking their potential. In 2023, Upwork’s talent community earned over $3.8 billion across more than 10,000 skills in categories including website and app development, creative and design, data science and analytics, customer support, finance and accounting, consulting, and operations.

Learn more at upwork.com.

Freelancer working at coffee shop

Going Deeper in Details

See how Upwork uses Promoted’s AI search results introspection tool to monitor, understand, and improve results.

Watch Introspection Video

Sree Kamireddy
Sree Kamireddy

Senior Director of Product, Next Gen Search

See Also

See how top marketplaces have increased conversions by +11% and continued to improve revenue by integrating with Promoted.

rvshare case study

RVshare

RVshare aims to enhance the rental experience for both entrepreneurs and renters by effectively matching renters with the ideal RVs...

outschool case study

Outschool

Outschool is an online learning platform that empowers kids ages 3–18 to learn on their own terms with the widest variety of teachers, topics, and classes to suit any interest...

Contact
Sales

Contact sales to license our cloud, self-hosted, or shared source open core solutions.