What could be wrong?

Trackcountry - sorry, backcountry.com

2015-01-08 / Found code

Statistics are important, sure. And analysis. And advertising. And customer insight, using predictive algorithms and big data to predict internet trends and behaviour. The webmasters for backcountry.com must be true believers in all of this. At least judged by the sheer number of external scripts and trackers I found in their page while looking into bug 1106810..

Here’s a potentially incomplete list based on listing the external scripts in the page I was served this morning, plus a quick skim through of the markup itself:

  • They naturally believe that getting the customer experience right starts with ForeSee, loading three script libraries
  • And they are also convinced that the Criteo Engine drives results and gives their customers personalized recommendations with these scripts.
  • It was slightly harder to find out what tracker this script is a part of, but it seems to be a library from CrazyEgg helping them see exactly what people are doing on the website
  • ..but I’m not sure if they’re satisfied with CrazyEgg after all, because look - there’s a good old Google Analytics script in the same page.
  • And because you can never have enough big data, here’s also a ScorecardResearch tracking script to ensure they get studies and reports on Internet trends and behavior
  • But why stop there when they can make every experience count by tracking and targeting customers with Optimizely? That’s two more JavaScript files to run.
  • And there are no limits to personalization, so why not be a customer of richrelevance too? They will certainly transform customer data into extraordinary customer experience with just two more scripts!
  • Speaking of tracking, there’s also the s_code.js library from the-tracker-formerly-known-as-Omniture, now Adobe analytics. This tracker is so invasive that I’ve analysed several problems where broken versions made entire sites unusable - either rendering them as empty white pages, or disabling all links and buttons. Let’s hope the version they have a copy of is reasonably bug-free..
  • And who doesn’t love saving time when managing reports and campaigns, as promised by Marin Software’s little tracker here?
  • And some inline code seems to come from Boomerang which I’m sure solves their digital marketing needs in just a few lines of JavaScript.
  • There’s a feature that pops up a “chat to a real person” - it seems to be LivePerson delivering lasting customer relationships - no wonder it takes quite a few scripts to do so.
  • They are a Google Trusted Store, which requires adding a couple of scripts.
  • Some more Google code running ads, I presume. There’s also a HTML comment with some inline scripts explaining that it’s going to “start Google Ad Services Other Conversion Tag”. And what about the JS for “Google Code for Smart Pixel List Remarketing List”? Well, let there be no doubt that Google believes their pixels are smarter than others. I got it right from the source.
  • Another inline comment is referencing “Nanigans Facebook Ads” - so let’s assume they are also taking control of their digital advertising with Nanigans.
  • Finally, why not let Mercent transforme their retail by tracking their customers too?

The mobile site also has code from New Relic - a company doing performance analysis for sites and apps. (Full disclosure: two of my ex-colleagues work there). I think I see some optimization potential here…

Also, the performance analysts might want to run this script:

document.documentElement.classList.length

and come up with some appropriate recommendations based on that number.

So we’re counting 27 scripts from 15 different companies crammed into that home page. This is the sort of sight that makes me feel I hope I haven’t looked into the future when reading their source code. But hey, I sure hope all that big data that is being generated works out for them. If you know any other good trackers they should add while they are at it, ping me on Twitter and I’ll sure let them know!

Web compatibility over a coffee - notes, not slides

Notes from meetup in Stockholm

Today, we had a small meetup in Stockholm, called “web compatibility over a coffee”. As it was in a café without projectors and such, I didn’t prepare slides or a presentation - but I jotted down some notes, gathered some bugs to look at and some screenshots. This is basically the basis for our conversation

Why web compatibility?

  • One altruistic reason - improve the web
  • One selfish reason - improve the UX for browser users

The web is the most widely used programming environment. The world’s greatest experiment in the capabilities and limits of human logic. A giant test suite for web browsers. And it takes two to tango: a web browser and a web site. A poor user experience is bad for a browser, whether it technically is our fault or not!

Some history:

Tech Evang component was added to Bugzilla in 2001.

Right now 1381 open bugs, of those 412 ASSIGNED

Around 14 000 bugs accumulated in component in total

Tech evangelism issues graph over time

Sounds like a lot of work.. why does this happen?

Well, it is a lot of work - and there are many reasons we may want to contact a site..

Old technology:

Browsers grow unexpected features sometimes:

The certainty of change:

Pick your poison - browser-specific code:

User-Agent detection:

(Sadly, Mozilla is also doing it - we’re making things worse, nudging some selected partners to look for a magic “.1” token in the version number. At the meetup, this led to a very interesting discussion about possibilities for adapting content to a device by measuring things like load- and scroll-performance, simplifying or enhancing the page correspondingly. It would certainly be interesting to see some experiments here!)

Sheer confusion:

So, where’s the frontier now?

Definitely on Mobile. The -webkit- mess. The iOS / Android duopoly which is rendering-engine-wise a monopoly. Desktop is however making a bit of a comeback due to mixed content blocking and generally stalling market share of Firefox

Any community forming?

This sort of work is both suitable and not for community building.

  • Suitable because there’s a lot of work to do, and much of it is just testing and outreach (i.e. doesn’t need extreme technical skills).
  • Suitable because contributors can learn a lot from analysing code and studying the existing analysis.

  • Not suitable because contributors tend to just pay attention to “their own” sites, do one or two bugs, and disappear
  • Not suitable because outreach is hard, and there is often a long time to wait before you see results.

For a wider reach, we’ve built webcompat.com. It’s becoming a cross-browser effort - all browser vendors struggle with old code, we all want the web to move forward. This helps but we still need more engaged users - and users who are prepared to stick around long-term, and work on issues across several countries, rather than just doing a few bugs or some outreach in a specific country - valuable though it is. There’s also arewecompatibleyet.com which is more Gecko-centric and presents Bugzilla Tech Evangelism data in a different way.

Can we automate any of this work?

Sure. You might think of simply doing wget / curl and diff commands. You’re wrong, it’s much harder than that.. Here are some of my site tests which I run with SlimerJS. We’re also working on using a volunteer-developed distributed infrastructure which can run tests continuously from several locations across the world. It will also support heuristics for discovering compatibility problems. This might take care of some of the boring work (testing to find problems, analysing simple issues) - and then we and the volunteers will handle the problems that require humans.

Thank you, Soumya, for organising the meetup!

CNN video redirects

Here are snippets from the JavaScript behind bug 970849, freshly served by CNN’s website:

(function(agent){
var index='/index.html',
match=/\b(iPad|iPod|iPhone|GoogleTV|Android|Kindle|Fire|Silk)\b/ig.exec(agent),

Hm, no Mobi there? What about a Tablet? Or even better - since this is client-side already - drop the names altogether and look at width and height to figure out how to adapt the content? The problem with name sniffing is that you’ll only be adapting to the few devices you already know the names of, not the millions of future devices you don’t. So, no surprise that this detection fails to handle Firefox OS smartphones, for example. Not to mention the Opera Minis and Mobiles, the phones shipping Access’s browsers etc.

device=String(((match instanceof Array)?match[0]:match)).toLowerCase(),

That instanceof call is a good idea though - as long as you know you might end up with device being “null”.

redirMap={

So, want to bet that this redirMap is going to contain rules for the “null” device?

'iphone': function(){

Snipping away some complicated rules for the iPhone..

'ipad':function(){

..and even more rules for the iPad.

}
return href.replace(path,path.replace(index,'/')+"standard.html");

This statement handles redirects for people who type cnn.com/video/ into those specific browsers this script cares about. It also does a special replace for people who type cnn.com/video/index.html. But what about users who type cnn.com/video without the final slash? Well, they don’t get to see any video.. Here’s what they get:

CNN videostandard.html 404 page screenshot

Onwards through the rest of the JavaScript:

},
'googletv':function(){

Oh, cute - there’s a special redirect for Google TVs. Somebody from the higher echelons of Google must be a Turner corporation board member or something..

}
};
redirMap['ipod']=redirMap['iphone'];
redirMap['android']=redirMap['kindle']=redirMap['silk']=redirMap['fire']=redirMap['ipad'];

And where is our “null” device handled, you might ask? Don’t be silly - being forward looking is so over-rated.

if(device in redirMap) document.location=redirMap[device]();

So this part just runs the hand-picked redirect rules based on the device you’re using (if they think it’s worth redirecting for)..

})(window.navigator.userAgent);

And this confirms it’s all built on top of the much abused navigator.userAgent property. As if we didn’t guess that from the first .match().

So, it’s official: iPhoneDroidKindle-level rich people who remember to end URLs with a slash is the only demographic the CNN cares about.

Yeah, what’s the point of a big world when you have a small mind?

IE girl

2014-09-23 / Site compat, Found code

SVG Girl website screenshot

Once upon a time, Microsoft wanted to promote IE9. I guess the IE6 stigma still stuck (9 is just a 6 upside down after all), so they were looking for a way to sprinkle magic coolness over the release. Enter SVG Girl - because what could sprinkle more cool than some anime with random strings of katakana? So there it is, a smooth, flashy and ultimately extremely odd animation of a girl who gets IE9 on her phone and gets … possessed. I can’t think of any other way to interpret that story - IE9 seems to enter her like an evil spirit and take control of her..

IE9 is history, but when you ship code on the web it tends to stay there. So this poor possessed girl is still out there promoting IE9. She’s even critical of visitors using later versions of IE - forever telling them that IE9 is the greatest thing ever. Hey - did IE9 even run on any phone in the Japanese market?

In any case, the page hangs in Firefox and Chrome. The simple reason is explained in bug 701672: when an element has id=myElm, window.myElm will typically refer to the element. IE turns out to be inconsistent for IFRAMEs, and window.myElm refers not to the IFRAME element, but to the window inside. It probably seemed like a good idea at some point..

The SVG Girl’s JavaScript expects IE’s implementation - or it expects other browsers to stick to their guns and not create references to tags with @id attributes on the global object.

Makes you wonder - was the coder possessed by IE9 too?

View test code on arewecompatibleyet.com

While the site arewecompatibleyet.com ships with test results from my web testing, it’s until now been somewhat of a mystery how those tests actually work.

Now, even casual visitors who do not want to search through big files of code on GitHub can get a sense of how the tests work. If you scroll down to the bottom of arewecompatibleyet.com’s front page, you will find a section listing all tests that output something which doesn’t match the state of the bug. If a bug is closed as fixed and the test fails, or the bug is open but the test passes, it’s listed.

There’s a tiny new feature: a link labelled “View test code” for each entry in this table. It brings up a small box where you can not only view the test code, but also copy a chunk of JavaScript that can run from the console.

For example, there’s an apparent regression on thestar.com, regarding bug 842184, no mobile content on m.thestar.com for Firefox on Android. Here’s the new feature at work, showing the test code (click to enlarge):

screenshot of AWCY site showing test code for bug 842184, regarding thestar.com site

Clicking in the grey area will select it for copying. The code you copy includes some “scaffolding” to make the test actually run - the code that is written for bug 842184 is the part inside the steps array:

(function(){
    var i=1, steps = [

function (){return mobileLinkOrScriptUrl() /*(regression test, expected to pass)*/ && hasViewportMeta() /*(regression test, expected to pass)*/}
]


    function doStep(){ if(typeof hasViewportMeta !== "function"){var s=document.body.appendChild(document.createElement('script')); s.src='http://hallvord.com/temp/moz/stdTests.js'; s.onload = doStep; return;};if(steps.length){var result = steps[0](); console.log('test step '+i+' says: '+result); if(result !=='delay-and-retry')steps.shift(); i++; setTimeout(doStep,300);}} doStep();})()

The instructions say you should switch User-Agent (either with a suitable extension or through about:config) to spoof Firefox OS (that’s actually a data bug since we’re dealing with a bug for Firefox on Android here) and load m.thestar.com. Clicking the site link opens it in a new window.

When the site loads, we open developer tools, go to the console and paste the testing code:

screenshot of console after pasting the test code - we see the output is false

As we see, the code returns false (aka ‘not good’). Now we can play around with the test code - for example by removing the code calling mobileLinkOrScriptUrl() which I highlighted in the screenshot above. And voila:

screenshot of console after pasting the edited test code - we see the output is true

So it looks like the test should no longer look for the ‘mobile’ keyword in file names. Unfortunately, testing the code we end up with against the desktop version of the site also returns “true”. To fix this test we should dig a bit deeper and look for some difference in the DOM. It’s sometimes hard to find stable features that are different - it can be an interesting challenge. Have fun if you try to help!