Which js-encumbered sites to fix in the first place

31 replies [Last post]
koszkonutek
Offline
Joined: 03/19/2020

As my project[1] that facilitates replacing websites' (nonfree) js with user-specified subsitutes is advancing, I am going to take a break from developing the actual browser extension and prepare some site fixes so as to avoid a so-called "infrastructure trap" where you focus on developing infrastructure for X and completely neglect X itself which was supposed to be the main goal.

Moving straight to fixing a complex site carries the danger of spending a lot of time on it. I would rather write some smaller and simple site fixes to serve as a good proof-of-concept to start with. Also, as this is supposed to be a showcase of how sites can be fixed, I'd rather focus on those whose JS is not free/libre in the first place. So, I am looking for recommendations on what sites should be fixed that:

1. Don't involve anything too complex like reCaptcha or stripe.js (time will come for those later)
2. Serve nonfree scripts in the first place (time for packaging libre js of free software projects like Etherpad will come later)
3. Are recognizable (e.g. I could make fixes for some polish sites, but who would care about those?)
4. Would be useful to you :)

[1] https://hachettebugs.koszko.org/projects/hachette

Avron

I am a translator!

Offline
Joined: 08/18/2020

One site I'd be interested in is https://www.leboncoin.fr (the good spot).

This is only of interest to people in France but perhaps of interest to everyone in France. This is free classified adds for anything, mostly selling second hand goods, but can be for anything. This is the best place for me to find cheap second-hand hardware that works with free software.

If working on something in French is too inconvenient for you, two sites on which I often found useful results when searching with duckduckgo for solutions to technical problems are https://unix.stackexchange.com and https://stackoverflow.com. Also, I find it impossible to check some information on free software visible on github withut running their javascript. I know github is not the service to promote but the services to promote run free JS and you have put that for later.

koszkonutek
Offline
Joined: 03/19/2020

Jahoti came up with a fix to make the cookie notice on Stack Exchange and Stack Overflow closable.

I gathered all the fixes we have so far in a repository[1] where they reside as .json files ready for import into Hachette. Jahoti's Stack* fix is also there if you want to try it out :)

[1] https://git.koszko.org/hachette_fixes_tmp

jahoti
Offline
Joined: 07/31/2021

As Wojtek noted, a (partial) fix for the cookie popups on Stack* is in the repository. If there's anything else that needs to be addressed on those sites, mention them and we'll take a look.

For Github and Le Bon Coin (does "coin" mean "spot" in French?), what exactly do you need implemented? Both are valuable projects, and I can look at both when there's time- peut-être j'apprendrai un peu de français!- yet without a clear scope for the work it will be practically unachievable.

Avron

I am a translator!

Offline
Joined: 08/18/2020

Thanks for your work.

Coin (to be pronounced kwain, in which "ain" is pronounced like in "I ain't got no money") can be translated to corner, area or "around here", depending on context. On Le Bon Coin, when I type something and click on "Rechercher" (search), if Javascript is not enabled, nothing happens. For a start, making that make a search would be nice. To be able to buy something, one still would need to sign in/log in in order to contact the seller.

For Github, on https://github.com/astrada/gapi-ocaml for instance, I see a number of small empty disks instead of contributors and empty rectangles next to each source file or directory. It would be nice to have the real information displayed. For some reason, I don't have that problem with https://github.com/astrada/google-drive-ocamlfuse (a nice tool to access a Google Drive without any Javascript by the way).

jahoti
Offline
Joined: 07/31/2021

Thank you for suggesting sites, and bringing up Google Drive OCamlFUSE (do you know if it works without an account?)!

I'll take a look at GitHub ASAP, and then Le Bon Coin. Thanks for sating my curiosity there, too- it's genuinely amazing to see how fast one word can effectively split in two.

andyprough
Offline
Joined: 02/12/2015

A few sites that I frequent where I have to allow js through noscript in order to use the content:

odysee.com - most of the good youtube video creators that I follow have put there content on odysee now. Enough so that I don't really use any other video site. It uses some js named odysee, bitwave, and lbry to function. This would be very useful, as this site continues to grow as the primary youtube alternative.

The internet archive - https://archive.org/index.php - I have an account there and post all of my libre GNU/Linux distro respin ISO's there for people to download. Very useful site.

phoronix.com - this site has good benchmark testing of the kernel and of various key components of a GNU/Linux system, along with news about package updates.

distrowatch.com - needs no introduction.

koszkonutek
Offline
Joined: 03/19/2020

Thanks for suggestions! Now, one more thing: would you be able to point out which parts exactly you want to be re-created on those sites?

For sites that cannot be viewed at all, what pieces of content are the most important to be viewable?

For sites that can be viewed without JS but have some functionalities broken, which of those functionalities are (the most) wanted?

If one functionality depends on another (e.g. being able to post depends on login), please mention both.

andyprough
Offline
Joined: 02/12/2015

odysee - can't search for or view videos without the js - this is, of course, of prime importance for a video hosting site

phoronix - can't log in and post comments without the phoronix.com js (according to noscript)

distrowatch - I think everything actually works at a basic level without js. This one should not be a priority.

internet archive - cannot watch video files without js. For example, my favorite Buddhism movie: https://archive.org/details/Samsara-PanNalin

andyprough
Offline
Joined: 02/12/2015

Also on phoronix - you can't view the benchmarking test results without allowing both the "phoronix.com" and the "openbenchmarking.org" js, according to noscript. Here's an example article from today: https://www.phoronix.com/scan.php?page=article&item=aocc31-gcc11-clang12&num=2

Without allowing the js, it's just a page with a few lines of text. With the js, you get all the pretty bar graphs showing the benchmark results.

koszkonutek
Offline
Joined: 03/19/2020

InternetArchive might actually be using free js (at least inline scripts are labelled as AGPL), so I won't prioritize it either. Video files are, however, available for download there. Is wrapping one of the links in a video tag sufficient? Or do you want something better (using other, higher-quality stream or sth?)

From barely looking at page's source I know how to get an .mp4 link for Odysee video and I can write code to put that in a video tag. Please don't count on fix for live streams for now. Odysee's search will need some more investigation.

Making benchmarks re-appear on Phoronix is trivial. Will do. As to login and posting - haven't looked into it yet

andyprough
Offline
Joined: 02/12/2015

You are awesome!!!

> "Is wrapping one of the links in a video tag sufficient? Or do you want something better (using other, higher-quality stream or sth?)"

Yes, wrapping the links in a video tag would do it. I can also just download.

>"From barely looking at page's source I know how to get an .mp4 link for Odysee video and I can write code to put that in a video tag."

That's incredible! Yes, search would be crucial, but this is a big start.

>"Making benchmarks re-appear on Phoronix is trivial. Will do. As to login and posting - haven't looked into it yet"

You are the best! Let me know if/when you'd like me to test things out?

koszkonutek
Offline
Joined: 03/19/2020

I started with Phoronix benchmarks. I attach a JSON file with the fix (named .txt because Trisquel forum disallows .json).

To install the extension, you can pull the source from [1], run `./build.sh mozilla` and load the resulting `build_mozilla/' directory as a temporary add-on.

For simplicity, I also put an xpi (which is no more than a zipped `build_mozilla/' directory with extension changed from .zip to .xpi) on our issue tracker[2].

To enable the fix, you go to the Hachette settings page (reachable through the pop-up after you click the hatchet icon on the top bar), click "Import", select the json file and proceed with importing all the stuff that Hachette found in it. Then, go to one of Phoronix articles with some benchmarks and see if it works.

== BEWARE OF OGRES PART ==
We're still working on some things that will make Hachette usable on daily basis. An option to allow scripts globally (right now Hachette blocks all scripts and only allows manual whitelisting of sites), some styling of the pop-up contents, changes to the handling of CSP headers, etc. Before we do all this, you'll have to deal with inconveniences. You might even find it better to run Hachette in a separate browser profile.
== END OF BEWARE OF OGRES PART ==

You can still give us feedback, though. And you can follow our progress on [3].

[1] https://git.koszko.org/browser-extension/
[2] https://hachettebugs.koszko.org/projects/hachette/files
[3] https://hachettebugs.koszko.org/projects/hachette/issues

AttachmentSize
phttps___www.phoronix.com____.txt 1.18 KB
andyprough
Offline
Joined: 02/12/2015

So with phoronix benchmarks, when Hachette says it "blocked" and then "injected" the "payload", then that means it's working I assume? Because I definitely do see the benchmark results.

koszkonutek
Offline
Joined: 03/19/2020

Is something else running that is blocking scripts? Like NoScript?

If not, can you open the developer tools and then the console and report if there's anything of interest there?

It might also make sense to go to add-ons and click to "Inspect" Hachette. The console there might give us some further clues

koszkonutek
Offline
Joined: 03/19/2020

Sorry for spam!

I mistakenly read "definitely do not see" instead of "definitely do see" and thought something is not working for you. Please ignore my post above.

Yes, it means it is working. Hachette IS blocking native scripts like NoScript or uBlock would.

koszkonutek
Offline
Joined: 03/19/2020

The package to enable video on Intenet Archive is here[1]. I tested it (only) on the Samsara page you linked.

An analogous work would be needed to enable audio, pdf and maybe other types of content. Please tell me when you need one of these. Before that - I shall move on to other sites.

BTW, in case any of you didn't notice it - most of the fixes are single scripts without dependencies. If there are any probles with Hachette, you can always just copy the script into your browser's javascript console (opened with Ctrl+Shift+C when browsing a page) and it should (mostly) work.

[1] https://git.koszko.org/hachette_fixes_tmp/tree/phttps___archive.org_details__.json

andyprough
Offline
Joined: 02/12/2015

With the internet archive json file, I keep getting a "Bad file :( Invalid JSON: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data" error when trying to import into Hachette

When trying to copy it into the javascript console, I get the same error message.

koszkonutek
Offline
Joined: 03/19/2020

Besides what I wrote in another post that you might have downloaded HTML instead of JSON, there's also 1 thing I should clarify: what should be copied into the javascript console is not the fix JSON itself, but rather the contents of its script. So, taking as an example my bank's credit card payment confirmation page fix (because it is shorter than the others), what you would be pasting into the console is:

/**
 * Copyright 2021 Wojtek Kosior
 *
 * Available under the terms of Creative Commons Zero.
 */

const submit_button = document.getElementById("submit");
submit_button.classList.remove("disabled");
submit_button.removeAttribute("disabled");

console.log(document.querySelectorAll(".noscript"));

for (const noscript_element of document.querySelectorAll(".noscript"))
    noscript_element.remove();

and not:

[{"scentrum24 Santander 3D-Secure":{"url":"","hash":"","text":"/**\n * Copyright 2021 Wojtek Kosior\n *\n * Available under the terms of Creative Commons Zero.\n */\n\nconst submit_button = document.getElementById(\"submit\");\nsubmit_button.classList.remove(\"disabled\");\nsubmit_button.removeAttribute(\"disabled\");\n\nconsole.log(document.querySelectorAll(\".noscript\"));\n\nfor (const noscript_element of document.querySelectorAll(\".noscript\"))\n    noscript_element.remove();\n"}},{"phttps://acsv.centrum24.pl/ACS/servlet/ACSAuthoriz":{"components":["s","centrum24 Santander 3D-Secure"],"allow":false}}]

nor the HTML from CGit

andyprough
Offline
Joined: 02/12/2015

Yup, that was the problem, I was downloading the html link instead of the actual .json file. I got Samsara playing in the Internet Archive video player now. Very cool!

All-time great movie love scene - Tashi meets Pema for the first time. Love at first sight - ahhh, who could resist falling in love with those eyes?

tashipemai.png tashipemaiii.png tashipemaii.png
koszkonutek
Offline
Joined: 03/19/2020

Phoronix relies on reCaptcha (at least registrations, but I prefer to only focus on login functionality once registrations are enabled anyway), so we would first need to port [1] to javascript. This is a bigger work and will only happen after we finish some crucial work on the extension itself.

[1] https://github.com/taylordotfish/librecaptcha

EDIT: Odysee update
I started working on Odysee and this is what I got so far:
https://git.koszko.org/hachette_fixes_tmp/tree/phttps___odysee.com.json
It can play a video like this one:
https://odysee.com/@osiixy:4/Big-Buck-Bunny-60fps-4K---Official-Blender-Foundation-Short-Film:5
Search is "half done". *Perhaps* I will complete it tomorrow. I am not doing any nice styling for now - sorry.

andyprough
Offline
Joined: 02/12/2015

This is all incredible!!! I should get time to try everything out this evening, I'll let you know how it goes.

andyprough
Offline
Joined: 02/12/2015

With the odysee.json, I also get the ""Bad file :( Invalid JSON: SyntaxError: JSON.parse: unexpected character at line 1 column 1 of the JSON data" error when trying to import into Hachette.

Probably something I'm doing completely wrong - let me know if it's obvious what error I'm making.

koszkonutek
Offline
Joined: 03/19/2020

I think I know what's the problem. The link I gave you leads to a cgit-generated HTML page of the JSON file with the fix.

You probably just wget'd or curl'd the link I gave you which I didn't assume you would. If you open it in a browser, you'll see a link to download the actual raw .json.

Anyway, the link for the raw JSON file you can just wget is as follows:
https://git.koszko.org/hachette_fixes_tmp/plain/phttps___odysee.com.json

Also, I noticed a mistake that causes the video not to appear on Mozilla-based browsers (I previously tested on Ungoogled Chromium). I'll push the fix within a few minutes.

EDIT: Another update. I implemented search for Odysee. Already pushed the enhanced fix to git.koszko.org (the same link as before). The fix now works as follows: if it detects the current page is an Odysee video page, it shows the video. Regardless of that, it always shows a search input. Search results appear without reloading, on the same page, below the search button. If my script at the start of execution detects the current page is an Odysee search page (i.e. one with url like https://odysee.com/$/search?q=something), it automatically performs the appropriate search as if you submitted it manually and shows the results. If you click on one of the results, it leads you to another page. If there aren't any fancy unicode characters in video name (not to be confused with video title), the video should appear there. For now, the fix only handles videos and not channels

andyprough
Offline
Joined: 02/12/2015

WOW!!! You have really done an amazing job!! odysee search box works, video playback works. I am very impressed, you know your stuff.

koszkonutek
Offline
Joined: 03/19/2020

Thanks for appreciation :)

I also know how to fix videos that don't work due to (probably) unicode emojis in their names (like [1]). That won't be much work. Await improved package in the git repo today (august 1st)

As to pcspecialist, another project member started working on it. We'll post in this thread if a fix gets written

[1] https://odysee.com/@EvelynJoy:4/funny-bunny-%F0%9F%90%B0-%F0%9F%92%97:f

EDIT: Odysee update.
You can now try the improved fix. Changes made:

* A workaround to display videos with unicode (e.g. emojis) in names has been added.
* Video descriptions now appear with their original formatting, with HTML entities properly decoded.
* Videos whose signing channel info is missing now also display properly in search results
* The "load more" button now gets disabled during loading of search results.
lanun
Offline
Joined: 04/01/2021

I would like to order from pcspecialist.co.uk but I cannot even see a price without JS.

Could you help me see the price and possibly order?

koszkonutek
Offline
Joined: 03/19/2020

Seems like another reasonable one to focus on. Showing the price should be doable (I assume you mean the summary price of all selected items). Ordering? Hard to predict before I or someone else actually tries to code the fix

lanun
Offline
Joined: 04/01/2021

> you mean the summary price of all selected items

Right, the price that should be displayed at the top of the page, and adjusted according to the selection below. Individual items prices are not always available, even with JS enabled.

> Ordering? Hard to predict before I or someone else actually tries to code the fix

No problem, being able to monitor prices would already be a nice step forward, thanks.

jahoti
Offline
Joined: 07/31/2021

Hi!

I've gotten a basic implementation of price handling working[1]; for privacy reasons the current setup has a requirement to press the "update prices" button at the top of the page to update prices, which I can remove if you want.

As for how to actually use the fix, see https://trisquel.info/en/forum/which-js-encumbered-sites-fix-first-place#comment-159234 .

[1] https://git.koszko.org/hachette_fixes_tmp/plain/phttps%20__www.pcspecialist.co.uk.json

lanun
Offline
Joined: 04/01/2021

This is truly awesome, thanks! And congratulations.

You might rename the add-on "Magic Hachette" :)

jahoti
Offline
Joined: 07/31/2021

Thanks! Perhaps we'll rename it once it can fix sites automatically. That would be a killer feature :)