Hero Image
dev

2021-04-20

Sniffing Login Requests

How to use puppeteer to sniff requests.

Written by

Image of Sam Larsen-Disney

Sam Larsen-Disney

Software Engineer at Amplitude

I have been pondering the idea of publicising my run data in an effort to get me to run more. If its public, I can easily be held accountable. I’ve been using the Nike Run Club since 2014 and have logged just over 200 runs over that time using the app. That’s a lot of data.

Sadly, even in a world where GDPR is a thing, Nike has not made it possible for people to download their own run data. I think this is in an effort to keep you using their app as having access to your run data would make it too easy to switch to another service.

The (Automated?) Journey for an Access Token

Thanks to @yasoobkhalid I came across this gist that explains how to hit the Nike API assuming you have a bearer token from the Nike site - they use the same auth service for both their site and the Nike Run Club app. The only problem with this process is that it requires you to go and login to Nike to get a new bearer token whenever you want to retrieve your data - Not cool. I want to be able to retrieve my data as part of my Gatsby build script every time I deploy my site and I don’t want to do this manually.

Time to whip out Node and see if I can hit the Nike Auth service with my login credentials. A simple POST request to their auth service. Or not. To cut a long story short, this failed. When trying to send this post request through node I got:

Copied!
<H1>Access Denied</H1>
You don't have permission to access "http&#58;&#47;&#47;unite&#46;nike&#46;com&#47;login&#63;" on this server.<P>
Reference&#32;&#35;18&#46;1e6cd417&#46;1571674983&#46;fd73639

Nike’s website seemed to be stopping me at every turn - not surprising really. I was probably missing a cookie that was needed for the request. Then I had a bright idea. Instead of trying to hit the auth API through a POST request, I thought I would instead see if I could intercept the login post request response using puppeteer as it would technically visit the site for me. Slower but I might have better luck! The code I came up with can be seen below:

Copied!
await page.goto("https://www.nike.com/gb/member/profile", {
waitUntil: "networkidle0",
})
console.log("👟 Navigated to Nike.com")
await page.click(".g72-menu")
console.log("🍔 Clicked on burger menu")
await page.click("#MobileAccountMenuHeader")
console.log("💪 Clicked Login")
await page.waitForSelector('input[name="emailAddress"]')
await page.type('input[name="emailAddress"]', process.env.nike_username)
await page.type('input[name="password"]', process.env.nike_password)
console.log("✍️ Filled In Form")
await page.click(".loginSubmit")
console.log("✅ Submitted Form")

This code goes through the process of logging into Nike. But we also need to listen for the response from the login request. We can do this by using puppeteer’s page.on method:

Copied!
page.on("response", async response => {
const url = response.url()
try {
const req = response.request()
const orig = req.url()
if (orig.includes("login")) {
const text = await response.text()
const body = await JSON.parse(text)
const status = response.status()
console.log("🎉 Found login response!")
console.log({ orig, status, body })
await browser.close()
callScript(body.access_token)
}
} catch (err) {
console.error(err)
}
})

The script takes 15 seconds to run, based on my internet speed - I can live with that, especially as I am only running this script when I build with an optional flag.

A small improvement

After further testing I managed to make it even faster by removing waitUntil: "networkidle0". I had added this to ensure the nav had loaded but it turns out that it was also waiting for all the resources to completely load the page. Instead I could just wait for the element I wanted to click await page.waitForSelector(".g72-menu").

This got the script run time down to 3 seconds on.a pretty average network speed. Nice.

About The Author

Image of Sam Larsen-Disney

Sam Larsen-Disney

Software Engineer

I enjoy teaching the next generation to code through my books, articles, presentations and at hackathons. You can learn more about me below.

Continue Reading