Support denyallow for removeparam

[Yuki2718]

Prerequisites

• I verified that this is not a filter list issue. Report any issues with filter lists or broken website functionality in the uAssets issue tracker.
• This is NOT a YouTube, Facebook or Twitch report. These sites MUST be reported by clicking their respective links.
• This is not a support issue or a question. For support, questions, or help, visit /r/uBlockOrigin.
• I performed a cursory search of the issue tracker to avoid opening a duplicate issue.
• The issue is not present after disabling uBO in the browser.
• I checked the documentation to understand that the issue I am reporting is not normal behavior.

I tried to reproduce the issue when...

• uBO is the only extension.
• uBO uses default lists and settings.
• using a new, unmodified browser profile.

Description

AdGuard URL Tracking Protection changed to denyallow format for the MV3 exception issue: AdguardTeam/AdguardFilters#220499 Currently it's not supported by uBO.

A specific URL where the issue occurs.

Anywhere one of those generic removeparam is used e.g.

https://www.irisplaza.co.jp/index.php?KB=SHOSAI&SID=209038&vc_lpp=MyY5NTM4YWVkNTE4ZCY2OGU1ZjQwZSYyODFmNWMmNjkzNTBlMGQmYU9YMERRQU5OWjQ3cG5jMENvb0FDZ3FLQnNPZ3d3JjQJYU9YMERRQU5OWjQ3cG5jMENvb0FDZ3FLQnNPZ3d3CTA4ODY1ODMyNzcwMjQyOTAzODI1MTAwODA1MTgwNQkJaHR0cHM6Ly9tYW5hZ2Uuc2FraWRvcmkuY28vP3A9Mjk2ODQ2MCZwcmV2aWV3X2lkPTI5Njg0NjAmcHJldmlld19ub25jZT0xNjcwODUzODE3Jl90aHVtYm5haWxfaWQ9LTEmcHJldmlldz10cnVlCXsiX2loIjoiYUhSMGNITTZMeTkzZDNjdWFYSnBjM0JzWVhwaExtTnZMbXB3TDJsdVpHVjRMbkJvY0Q5TFFqMVRTRTlUUVVrbVlXMXdPMU5KUkQweU1Ea3dNemciLCJfdXJpIjoiaHR0cHM6XC9cL3d3dy5pcmlzcGxhemEuY28uanBcL2luZGV4LnBocD9LQj1TSE9TQUkmU0lEPTIwOTAzOCJ9

for the vc_lpp rule in the list.

Steps to Reproduce

1. Visit the link with AdGuard URL Tracking Protection enabled
2. See vc_lpp parameter is not removed

Expected behavior

The vc_lpp parameter should be removed except for booklive.jp which I currently can't find a link with the parameter.

Actual behavior

The rules are rejected.

uBO version

1.68.0

Browser name and version

Chrome 143.0.7499.41

Operating System and version

Windows 11

[gorhill]

I don't understand the changes.

The original filter, $removeparam=vc_lpp,domain=~booklive.jp means: "Remove vc_lpp parameter when context is anything except booklive.jp"

The new filter, $denyallow=booklive.jp,removeparam=vc_lpp means: "Remove vc_lpp parameter only when network request is booklive.jp" -- essentially the opposite of the original filter.

The new filter fails because denyallow requires that there is a domain condition, otherwise that sort of rule would essentially break the internet, this was a requirement back then to avoid serious breakage. The same can be accomplished with $to=booklive.jp,removeparam=vc_lpp. Still, I do not understand the change as the new filter, corrected or not, does the opposite of the original one.

The new filter should be $to=~booklive.jp,removeparam=vc_lpp to be equivalent of original one.

[Yuki2718]

The new filter should be $to=~booklive.jp,removeparam=vc_lpp to be equivalent of original one.

@slavaleleka Can this be addressed in FiltersCompiler?

[gorhill]

Even AdGuard's documentation mentions the need for domain=: https://adguard.com/kb/general/ad-filtering/create-own-filters/#safari-converter-lib

But still, AdGuard's implementation is not equivalent to uBO's one. There are no exception filters created internally by uBO to implement denyallow. AdGuard internally creates exception filters, which could end up excepting unrelated filters. (Never mind, seems to apply to Safari converter only) In uBO, denyallow= was created before to=, and when I added support for the to= option (to be on par with DNR syntax), denyallow= simply became a subset of to=, except that denyallow= still enforces the domain= requirement, to= does not.

[uBlock-user]

to= supersedes denyallow, so this is not needed.

[gorhill]

Currently AdGuard URL Tracking Protection is severely broken for uBO with these changes. Mainly, AdGuard created their own denyallow semantic beyond the one originally implemented in uBO, and the changes here rely on that specific AdGuard implementation. I rather not change our implementation of denyallow, which predates AdGuard adding support for the option, and anyway it's not something that can be changed within hours to fix the current breakage. At most, for the future I am open to create a new option for convenience, such as not-to=, for a list of hostnames to exclude from the filtering (and maybe not-from= for consistency).

Given this, I want to fork AdGuard URL Tracking Protection into our own maintained list. Any objection?

[u-RraaLL]

Is this breakage the consequence of the change? https://www.reddit.com/r/uBlockOrigin/comments/1psjmzd/err_too_many_redirects_most_likely_because_of/

Given this, I want to fork AdGuard URL Tracking Protection into our own maintained list. Any objection?

Do you mean uBO URL Tracking Protection or keep it as AG, but revert the change they've done and keep updating using the prior syntax?

[stephenhawk8054]

Given this, I want to fork AdGuard URL Tracking Protection into our own maintained list. Any objection?

Yeah no objection, I thought about doing this myself yesterday too. An important point I always feel not right is that removeparam is meant to act on requests, not site context, hence it feels more natural to use it with to in most cases than domain.

[Yuki2718]

No objection, but I think there's no breakage currently but only those generic rules are discarded. Maybe we can follow EasyList – Social Widgets etc. like "AdGuard - Tracking parameters".

[gorhill]

New file at: https://github.com/uBlockOrigin/uAssets/blob/master/filters/privacy-removeparam.txt

I will wait that it propagates to uAssetsCDN before modifying assets.json et al.

[Yuki2718]

@gorhill uBOL 2025.1229.1729, but not uBO, removes from parameter when accessed https://www.nicovideo.jp/watch/sm45764446?from=100 with AdGuard/uBO – URL Tracking Protection enabled, despite there's no generic or specific rule to remove from in the list. The closest will be $removeparam=from,domain=anime.nicovideo.jp|dic.nicovideo.jp but it should not be applied to the url.

[gorhill]

The related filter is ||jlcpcb.com^$removeparam=from,to=~passport.jlcpcb.com, which for yet unknown reason ends up as the following DNR rule:

{
  "action": {
    "redirect": {
      "transform": {
        "queryTransform": {
          "removeParams": [
            "from"
          ]
        }
      }
    },
    "type": "redirect"
  },
  "condition": {
    "excludedRequestDomains": [
      "passport.jlcpcb.com"
    ],
    "resourceTypes": [
      "main_frame",
      "sub_frame",
      "xmlhttprequest"
    ],
    "urlFilter": "^from="
  }
}

When I generate DNR rules from within uBO, the rule is correct:

  {
    "action": {
      "redirect": {
        "transform": {
          "queryTransform": {
            "removeParams": [
              "from"
            ]
          }
        }
      },
      "type": "redirect"
    },
    "condition": {
      "excludedRequestDomains": [
        "passport.jlcpcb.com"
      ],
      "requestDomains": [
        "jlcpcb.com"
      ],
      "resourceTypes": [
        "main_frame",
        "sub_frame",
        "xmlhttprequest"
      ],
      "urlFilter": "^from="
    },
  }

Somehow the requestDomains property is dropped somewhere.

[gorhill]

Ok it just happened that I built uBOL just before this commit, which fixed the filter: uBlockOrigin/uAssets@38bb7ef

---

Anyway there is a new version 2025.1230.xxxx submitted to the Chrome store, which should contain the fixed filter