[MUSIC PLAYING] CHENEY TSAI: Hey, everyone. My name is Cheney. RAMYA RAGHAVAN: And I’m Ramya. CHENEY TSAI: So we work
on a technical consulting team at Google, and we
get to see companies all around the world push the
boundaries of the modern web. RAMYA RAGHAVAN:
And while you might have heard some
great sessions today talking about different
modern web technologies, we are here to focus on the
real-world challenges faced by e-commerce companies
in implementing these technologies. CHENEY TSAI: So since we’re
talking about the real world, a real-world metaphor. Maintaining a website is a
lot like maintaining a house. This right here is the
house that I grew up in. It’s probably not like the
house that you grew up in, just like the website you work
on is different from, say, the person over there. But some things are the same. You have maybe a
place you sleep. Maybe there’s a dining room. Then something
breaks, like the AC, you might say,
well, I can fix it. And your friend might
say, no, you should just call somebody, all right? Over time, when
you moved in, when you moved to that
new platform, it might look like the room
over here on the left. But technical
debt, feature creep because Black Friday and the
holiday season is coming up, you’re trying to get things
in, and you get kind of messy. And I think that a house
is a great metaphor because it’s like an
e-commerce site these days. It’s not just about landing
in one place and you’re done. It’s about the journey. You’re going from room to room. You’re going from a
page to another page, from the home page to the search
page to the checkout page. And if the room is like this,
you get kind of confused. You don’t want your user to feel
confused and get lost in it. So it’s time to remodel. Maybe you need to
upgrade your foundations. Obviously, to
remodel, we’ve talked about a lot of different
things throughout today and other talks,
things like that, and like all tools
in the tool box, things like Progressive
Web Apps, AMP, Google sign in, Google
Page– they’re great tools. But just like I
can’t go to the store and buy a power drill that
I saw a great commercial of, and be like, I’m going
to nail this remodel. I could very well
destroy everything. So like all tools,
just because they fit– they’re kind of
like puzzle pieces. The two pieces fit, but you
might not be actually making progress towards a big picture. So what’s the big picture? So at Google, we like to
say, focus on the user and all else follows. And so we really think
that that full picture you’re building
towards, whatever you choose to do in
the e-commerce journey, you need to improve
user friction. Think about if you’re trying to
add service workers, if you’re trying to try some of the
cool, new APIs, and like, WebXR and things like that. How is your solution actually
improving the user’s journey? Because there’s no
silver bullet here. To talk about this is Ramya. You had an example? RAMYA RAGHAVAN: Of course. Saatva is a company that
sells luxury mattresses. And they wanted to ensure
that their users have a very smooth and fast
online shopping experience. They decided to use AMP as
a technology to enable that. In their first iteration
of their product details pages where they created the
AMP portion of that page, the user had actually to
click from the AMP page to the product details
page and then click again to get to the cart page. They realized that this was
not the best user experience. So in their second iteration,
they changed the design such that the user can see the
full details of the product within the AMP page and be
able to click to add to cart, and even see the cart
from the same page. This resulted in a reduction
of friction for the user, going from three taps to
two all in the same page instead of three pages earlier. The results were astounding. Between the two
iterations of AMP, they saw a 50% improvement
in conversions. CHENEY TSAI: Thanks. So it’s not just AMP. If you’re creating things
like a Progressive Web App, you might run into this too. So here’s an example
from Expedia. Expedia has been working on a
brand new front-end platform. It’s four times faster on
slow 3G, on lower end devices. It’s really great. And their CEO even recently
said in their earnings call that speed benefits
deliver conversion benefits. Usually, we’ll say,
we’re going to stop here. Great case study. Instead, we’re going to go into
an interesting learning they had as part of this process. So Expedia, like many of the
sites that you might work on, is a large e-commerce site. They’re not just going to move
from one old platform that’s clearly working over to
a new platform overnight. They need to do this
in a piecemeal way because maybe some teams
are on a new platform. They have to socialize
it to other teams. And so in doing
that, they actually learned an interesting lesson. Users value consistency. So some users during
the migration process, they landed on the
old experience. And then, after
clicking around, you get put into the new
experience that’s faster. And they might move back
and forth a couple of times. That population, to no surprise,
actually probably did poorer than if you just stayed in
one experience or another. It’s very much like,
if I was up here, and I was wearing one brown
shoe and one black shoe, you probably couldn’t
take me quite as seriously as you’re taking me right now. Hopefully, it’s on
the serious side. So I think users value user
journeys that are consistent. So make sure your solutions,
not only in that page you’re building, but
in the entire journey and how you’re measuring it
is actually reducing friction. Of course, what
I’m saying, though, means that if your journey
needs to be consistent all the way from, say, the entry
point for the search results page to checkout,
you’re probably going to run into a
lot of challenges. So when we work with a lot
of e-commerce partners, we run into three
main challenges. Number one is something
that many of you probably think about,
organizational alignment. Then you get to
technical approach, and then finally, measurement. And so we’ll talk
about that today. RAMYA RAGHAVAN: Of course. Let’s talk about
the first challenge, organizational alignment. When I talk to people
in e-commerce companies and ask them, do you think
investing in performance projects to improve user
experience will help improve your core business metrics,
the answer is an immediate yes. However, when it comes time
to prioritizing performance projects, there’s
always several hurdles– lack of time, lack of
resources, you name it. One of the key ingredients
that’s missing here is organizational alignment. Performance is not just
an engineering priority. Why? Consider this. Your content creation team
wants to generate great content for the users to consume. However, they
might create images that are too heavy
or not optimized because they are not going
through the image optimization layer. The marketing team
spends several dollars to bring traffic to the
site and care deeply about the user experience. However, they might
also have tags that are either old or
redundant or obsolete. The design team cares very
much about that end-to-end experience, but they might have
some really cool new features that do not use
shared competence. In each of these
scenarios, one thing is clear, which is
that performance needs to be a priority
among every team that affects the website and
cares about user experience. Let’s look at two
approaches, a top-down and a bottom-up
approach, in order to create organizational
alignment. And we’ll do this using
two examples of partners, 1-800-Flowers, which is a
global flower brand, and then JustFly, which is a
Canadian travel agency. 1-800-Flowers believed that
the modern marketing department is really marketing
plus technology. This is a very big
shift in mindset. And what they did was to break
out of organizational silos and create a
cross-functional team that comprised of product,
marketing, design development, and analytics. This was led by the
CMO of the company. And they were given a
very clear shared goal, which was to drive
growth for the company by investing in the emerging
product experiences. This allowed this
cross-functional team to work really closely to
invest not just in one or two, but almost the entire spectrum
of modern web technologies. And as you can see, this
yielded some great results. Seeing a 50% uplift in
conversions does not come easy. Let’s look at the
other example, JustFly. Even if you don’t have
a top-down mandate, you can use data to
kind of prove your case and get that consensus. JustFly did just that. They started off with
bite-sized proof of concepts and experiments and
got the consensus amongst their management
and cross-functional teams to invest in a [INAUDIBLE]. They started off with
this high-level goal of improving the booking
experience on their site. And they used data
and experiments for just about everything, from
testing different GSA libraries to testing different
checkout flows and design. And also, they had [? at ?]
[? the ?] [? home ?] [? screen ?] icon being
[INAUDIBLE] by a few users to see how those users behave
differently from others. And with all of this
data, they not only got the buy-in from
product and marketing, but they also got the head count
they needed to build the PWA. And they merged the code
bases of two of their brands so that they could
launch both of them as a PWA in the same
time frame that it would have taken for one. Now, that’s efficiency. And as you can see, with a
35% improvement in bookings, this is a great result for them. All in all, whether you
use a top-down approach or a bottom-up approach, it’s
important to keep in mind that organizational alignment
is key to not just prioritizing your performance
projects, but also to gain the right visibility
so that everybody that cares about user experience
also cares about your project. CHENEY TSAI: Well, once you
get the organizational buy-in, you still have to
build the project. So we’re not here today to
talk about all different ways to build a great Progressive Web
App or a modern web experience. We’ll share a little bit
of these successful traits we see of good projects
we see out there. One big pattern is
that we see success in having a long-term
vision and then having achievable goals
towards that vision. To give an example– so I was recently in
Spain with my wife, and we visited la Sagrada
Familia over in Barcelona. I think it’s a great
example of this. This has been under construction
for five generations. And yet, it still adheres
to the original architect, Gaudi’s, original vision. Despite that, even
though it’s still under construction to this
day, it opens its doors to 2.5 million tourists a year. It’s actually very much like
some of the e-commerce sites that you have. On Black Friday, not only are
people going to check out, you probably are running
like 2.5 million A/B tests through some platform. So I think it’s
really interesting to think about that. And a lot of teams have
that long-term vision, and they often tell
me that, yeah, we want to get that
time-to-interactive to under five seconds. But oftentimes,
you find yourself, we ran Lighthouse and we’re
really far away from it. And what happens is, well,
we’ll get there someday. And then there’s nothing
achievable to get there. It’s just something you see off
in the distance in the horizon. Oftentimes, they come back
and say, well, we’re just going to have short-term goals. And this is great. They have sprints or
everything to get quick wins. And not to say quick
wins aren’t bad, but oftentimes, you kind of get
into this internal cycle of, well, there’s a deadline here. It’s something we can achieve
before the holiday season. And then that’s all you do. You go in the circles or you
regress after it a little bit. It’s really important that you
have both a long-term vision and a short-term
goal and you make sure that they are
actually related, that the short-term goals is
making meaningful progress towards a long-term vision. Let’s make this a little
bit more concrete. We all like to say it up on the
stage with the other speakers that one of the visions
is to improve speed. Obviously, what does that mean? There’s a lot different
ways to measure. So let’s go back to what I
was saying earlier from a user perspective. Improving speed might be to
focus on user navigations. So when we say you want to
optimize your cold page load, or maybe your return page load
to see if you’re using caching, or if you’re comparing
single-page app to multi-page app,
a lot of things we’re talking about that
ease, that latency between me clicking on the user
interface and the response in your UI of
something showing up. So let’s talk about
speed in that regard. So to go back to the
house really quickly, I think doors are a
great kind of example of what navigation is. An e-commerce site–
it’s like a house. You move from room to room. You might come in through the
back door, the front door, or something like that. And throughout the
process, to complete the journey in the house,
like to complete the journey on your website, you
move from room to room. On Thanksgiving, you’re holding
the hot turkey in the kitchen. Do you want to mess around
with the door on the left here where you have
to find one key and then grab another key to get
through the door in the back? It’s probably not
the door you want. Likewise, there are
probably experiences in your e-commerce journey
where you don’t want the user during sorting,
filtering, or searching to have to kind of pay the
cost of that locked door. There might be a good place for
you to have a door on the right here. I’m not saying that it’s always
client-side routing or anything like that, but there are
different techniques of making that as seamless as possible. So let’s jump into three
examples from Walmart, eBay, and Airbnb, who are in different
parts of this modern web journey, and kind of dive
into how they’re breaking down this process. Walmart’s a great example– extremely leading
e-commerce site, and it recently made a North
Star Goal to reduce time to interactive by 70%
on low-end devices, such as the iPhone
6s and Moto G4. And this is an extremely
aggressive goal to going back to what I was saying earlier. But I liked that they were
able to kind of make that goal. So they organized a bunch of
product and engineering teams to make meaningful
progress towards that. And then in the last couple of
months, they got 25% of the way there with improvement
on some pages. But just because you make
good progress from 0% to 25% doesn’t mean just making
quick wins or the same process is going to take
you from 25% to 70%. But that work is important
because in that work they did, they were able to show
significant business improvement and use
that to socialize it across their teams. But what do I mean by
going from 25% to 75% might not be quite as simple? Take this. It’s not really my kind of
room, but let’s say it is. I could organize this room by
just taking some books out, and reshuffling, and making sure
they’re all in the same order. But if I was going
to take that strategy and do that for
everything in this room, I don’t know if I’ll
make meaningful progress and finish in time. If I’m going to
clean up this room, I might actually
say, well, look, this is kind of how I would
like one shelf to look like, but let’s kind of take
all the books away, move the shelves
around to actually get a nice looking place. Likewise, what Walmart’s
doing is that because they had a North Star Goal and
they made short-term progress, they’re now able to start
thinking about, well, what other larger shifts
can we take to get to the goal we want to do? Is it building a
Progressive Web App? Is it adding service workers
in some meaningful way? I think we’re really
interested to see kind of what they’re going
to work on in the next year. Speaking of service
workers, eBay is actually using one of the
service worker techniques to improve user
navigation on their site. So in one of the
earlier talks, we talked about how you
can use service workers to improve user navigations. There’s lots of different ways. eBay is just using
one potential way, and they’re testing
this right now. So they’re doing predictive
prefetching of listing pages. So for example, right now,
if I go to eBay, and say, I’m looking for a camera
for Christmas for my dad– and dad, you didn’t hear that. I don’t think you’re tuning in. So right now, if I see a
camera in the search results, I have to wait until I
click on the listing, and then the browser makes
another HTML request, and I’m waiting
for that as a user. And depending on what
connection I’m in, that could take a long time. Well, eBay’s like, if we could
predict with high accuracy what you’re likely to click
on– you’re going to click on this camera right here. What if we post message
to this service worker that particular file you need. And then the service worker
is pulling in the background while I’m considering
which camera to click on so that when I do
click on it, well, it’s already there on my phone
so that when I click through, it’s as instantaneous of
a navigation as possible. Of course, what if you
don’t have a service worker architecture yet
already in place? You’re waiting on Q3
for that to be done. Or maybe some of the quick wins
for initial page load might not be there and you’re
kind of stuck there. Because you have a long-term
vision of improving user navigation, you might run
into a wall at some point of a short-term goal. Because you have
that setup, you can start thinking about
different ways of getting to that long-term vision. And this is kind of where Airbnb
is kind of thinking about. So Airbnb is like, well,
sometimes rebooting that JavaScript location
on every single page might be a cost we don’t want
every single page load to be. So they saw that,
what if we can only update certain
parts of the page, say, in the search results so
that only that refreshes, only that shows up when the
user clicks on something? In doing this, they
actually measured the user-perceived
speed of that load. And they saw that loading
the search results page in that manner was a seven
to eight time improvement. And so let’s say they
start diverting more user traffic to take advantage
of client-side routing so that now 88% of
search results loads are leveraging the seven to
eight times speed improvement. Because at the end
of the day, it’s the user’s journey that matters. Hopefully this is
a quick example of how you can keep an eye on
long-term vision, wherever that may be, set short-term goals,
and realize that there are a lot of different
ways when you hit different walls in those
short-term goals of getting to that long-term vision. RAMYA RAGHAVAN: Let’s talk
about the third challenge, measurement. Let me ask you something. Please raise your
hands if you’ve ever worked on a project
with great potential to improve user
experience, but the plug got pulled at some point because
you couldn’t prove the impact. Anyone? That’s quite a few of us. And this is certainly
not uncommon, right? Even if you have a great
organizational alignment and a good technical
approach to your project, if you don’t have the
right measurement strategy, it could still fail. Because of this, even though
this is the third challenge we are covering here today,
this is one of the first things that I’ve seen
successful companies do in terms of thinking about
their performance project. A good measurement strategy is
one that is shared, automated, and actionable. What do I mean by that? It needs to have shared metrics. To Cheney’s earlier point, when
you have the North Star Goal, you need to have
specific metrics that are mapped to that North
Star Goal that are also shared amongst people that care
about that user experience. Having these shared
metrics is really crucial, and you have to
define it in a way that people understand
exactly what those metrics are and how they are calculated. The second piece here
is automated tools. These tools are necessary, not
just to measure these metrics, but also monitor them over
time so that you can gain real good insights from them. And once you have
those tools, it’s important to understand, what
are the actionable next steps? Now that I have this
insight, what changes can I make in order to
meet that North Star Goal? Let’s dive into shared metrics
a little bit deeper because it’s often not as well understood. When I talk about mapping
goals to specific metrics, we have these four
big milestones when we talk about a
user journey in terms of acquisition, engagement,
conversion, and re-engagement. Let’s focus more on
the conversion piece. Let’s say you really wanted a
seamless checkout experience for your users. One way to measure that is to
think about the average time that it takes for, let’s
say, 90% of your users to go from that cart page to
the order completion page. Understanding from
a user perspective how easy or difficult
it is for them to convert when
that intent is clear is a good way, potentially, for
measuring whether you’re having a seamless checkout flow. Another important point to
consider is third parties. Third parties are an integral
part of that user experience on your site. And therefore,
they should not be ignored when it comes to
measurement strategy either. So one way you can hold
your third party accountable is to say that the
server response time should be under a certain
budget that you set for them. All in all, talking
about budgets, you’ve been hearing this
term of performance budgets throughout various
sessions today. A performance budget
is a framework that allows you to understand
what changes represent progress and what changes
represent regressions. And it does take into account
the set of shared metrics, and you have budgets for
each of those metrics, which are shown in this automated
tool and made actionable. Let’s actually
take a look at how three different
e-commerce companies took this idea of a
performance budget and implemented it
so well that they have seen some amazing results. Are you ready? OK, Cdiscount is a French
leader in e-commerce that had a problem that is
not very unlike many of us in e-commerce. They saw a big chunk of
their traffic on mobile, but they didn’t see that
the mobile sales were at a similar ratio. They realized that this is
not an easy problem to solve, right? So they broke
organizational silos, and they empowered
their marketing team to run deep analysis
and come up with a set of shared best practices. They then implemented these
best practices as budgets within their tool. And using the integration
between SpeedCurve and Slack, they ensured that when
the budgets were not met, automatic alerts would be sent
into the Slack channel which was shared amongst all the
cross-functional teams. By collaborating in making all
of these different changes, they saw some really,
really good results. Their mobile website
conversion rate was 50% higher than their
native app, which has a 4, 4 and 1/2 star in
different app stores. This is something that all of
us really can relate to, right? It’s not easy to get there. Let’s take another example. Carousell, which is a
Singapore-based online marketplace, used this idea of a
performance budget because they really wanted their users in
southeast Asian countries where the predominant
network speed is 3G– they wanted all of those users
to have a great experience on their site. And so they started
off with this kind of performance budget. Note that the first line here
is about the critical rendering path resources bundle
to be under 120 KB. This is irrespective of
whether that bundle includes first party or third
party because they had this commitment
that that bundle needs to be really small. They were so intent upon
following this performance budget that they
included this as part of their continuous
integration process, such that the code
could not be merged until this bundle or
this budget was met. In order to do that, it was
not an easy thing at all, so they really had to
collaborate very, very closely with their design, and
marketing, and analytics teams, and that really paid
off very well, as you can see in this next slide. Not only are these
metrics really good, but there’s something
here for every team that was part of this project. The product team cares
about the page load time. The marketing team cared
about organic traffic. The ads team cared about
the click-through rates. And the transactions
team cared about how many buyers and sellers were
able to chat with each other to complete their transaction. So these were great
results all in all. Let’s look at a third example. As you heard earlier today about
Wayfair, the home goods company here in the US,
that really wanted to improve user experience
and make sure that they sustained that improvement. When they first started off with
the performance budget, which they said was one of
the most crucial parts of the measurement
that they did, it did not take off that much
because these budgets were understood, but it was not
very clear for everybody. Well, they made so many
changes towards optimizing their [? inside, ?]
they said that one of the most important
things they did was to socialize their budget. And the way they did
that was to make it clear using performance indicators,
as you see here in colors. So this is how it worked. They had a budget, and they had
an acceptable variance of plus or minus 10%. So if you were within that
range, you’re in orange. If you exceed that
budget by more than 10%, then you enter the red area. And if you’re better than
the budget by more than 10%, then you’ve hit your goal. Congratulations, you’re a green. They also had a competitor
line, just to make things fun. This is how it looked
for all internal users when they looked at
their specific pages since the budgets were
set at a page-type level for every route in the site. So the moment you look at your
mobile site within the company, you were able to see
these indicators that told you whether you are in
the red, green, or orange. As you can see
here, the way they set these budgets was the moment
you hit the goal reliably, your budget, and
therefore, goal, which was always 10%
better than the budget, tightened up a little bit. You can see that over here,
where once you hit the goal, the bar was raised, so
to speak, and it actually tightened up over there. By the way, isn’t
this the kind of graph we all want to see, going from
2.3 seconds to 0.9 seconds? That’s really awesome. This is a performance portal. As you can see, in
this portal, they had a clear set
of shared metrics. Not just the metrics, but
also the page level indicator, and they have a call to
action at the bottom for users to basically reach
out to a Slack channel if they had questions
about these metrics. Whether you’re trying to use
a performance budget idea or trying to use the general
idea of a shared metrics automated tool and
actionable next steps, thinking about your
measurement strategy at the beginning
of your project is crucial to the success
of the project. Another thing to remember
about performance budgets is that it needs
governance to think about the metrics at
a page or route level, and also to maintain those
budgets and metrics as moving targets as you go forward. It also needs to be
enforced in a way that everybody can come together
to meet or beat those budgets. And that makes it
very, very successful. CHENEY TSAI: We would
love to go deeper into some of these partner
stories, but true to form, we might be slightly above our
performance budget for time. So hopefully you learned
something from today’s talk. I think we want to focus on
making sure your tools actually improve the user journey. And along the way, you might
run into these three problems, org alignment, technical
approach, measurement. This isn’t anything
new to us, and we hope some of these
partner examples help you move a little
further along on that journey. RAMYA RAGHAVAN:
Which means you need to do something differently
the next time you work on a performance project
to build consensus and visibility for
your project, to cross those technical hurdles to
reach the North Star Goal, and also, not just
to prove the impact, but also to sustain
the improvement that you’ve worked so hard for. When you have all of
these considerations for your project, I’m sure that
working on performance projects can be very exciting
and rewarding, as many of these
partners would attest to. I’d like to say a special thanks
for all of these partners that allowed us to share their
stories with you today and many others who’ve
influenced this talk in some way or another. We would love to
hear more from you and your specific
challenges when you’re trying to adopt
modern web technologies. Please come find us at
the [INAUDIBLE] section during the break. Thank you. CHENEY TSAI: Thanks, everybody. [MUSIC PLAYING]