fix Pleroma search API incompatibility with empty `limit` param #977

snarfed · 2020-09-27T23:31:17Z

Pleroma supports the Mastodon API, so a number of people have already signed their Pleroma accounts up for Bridgy via the Mastodon support. And it works!

...at least, mostly. Publish seems pretty functional, and listen (aka backfeed) has worked before, but right now it's hitting the API incompatibility below. Evidently Mastodon's /api/v2/search endpoint handles a blank value for the limit query param ok, but Pleroma doesn't. cc @jk-na etc.

I 2020-09-27 22:40:01.151421 requests.get https://toot.icyphox.sh/api/v2/search {'params': {'q': 'icyphox.sh', 'resolve': True, 'offset': 0, 'limit': ''}, 'headers': {'Authorization': '...'}}
I 2020-09-27 22:40:02.334777 Redirected to https://toot.icyphox.sh/api/v2/search?q=icyphox.sh&resolve=True&offset=0&limit=
E 2020-09-27 22:40:02.338004 Error 400, response body: '{"error":"limit - Invalid integer. Got: string.","errors":[{"message":"Invalid integer. Got: string","source":{"pointer":"/limit"},"title":"Invalid value"}]}'
Traceback (most recent call last):
...
  File "/srv/tasks.py", line 86, in post
    self.poll(source)
  File "/srv/tasks.py", line 134, in poll
    links = source.search_for_links()
  File "/srv/mastodon.py", line 148, in search_for_links
    return self.get_activities(
  File "/srv/models.py", line 393, in get_activities
    return self.get_activities_response(**kwargs)['items']
  File "/srv/models.py", line 387, in get_activities_response
    resp = self.gr_source.get_activities_response(**kwargs)
  File "/layers/google.python.pip/pip/granary/mastodon.py", line 197, in get_activities_response
    statuses = self._get(API_SEARCH, params={
  File "/layers/google.python.pip/pip/granary/mastodon.py", line 120, in _get
    return self._api(util.requests_get, *args, **kwargs)
  File "/layers/google.python.pip/pip/granary/mastodon.py", line 135, in _api
    resp.raise_for_status()
  File "/layers/google.python.pip/pip/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://toot.icyphox.sh/api/v2/search?q=icyphox.sh&resolve=True&offset=0&limit=

The text was updated successfully, but these errors were encountered:

jk-na · 2020-09-28T00:34:19Z

I only just noticed earlier today that I wasn't seeing feedback via pleroma reflected on my site since I updated pleroma to the 2.1.1 release a few weeks ago. (Edit: possibly 2.1.0 as there seems to have been many api updates there: https://git.pleroma.social/pleroma/pleroma/-/blob/stable/CHANGELOG.md )

I'll track down some working and non-working URLs tomorrow and raise a pleroma ticket.

snarfed · 2020-09-28T01:15:07Z

cc @lambadalambda @rinpatch @feld, just FYI, feel free to redirect to whoever on the Pleroma team works on the API!

jk-na · 2020-09-28T08:19:01Z

This is the log from the last successful interaction from pleroma to my server:
D 2020-09-08 09:42:07.925561 Params: [('response_key', 'agdicmlk...')] I 2020-09-08 09:42:08.044517 Source: @jk@social.nipponalba.scot (Mastodon) @jk@social.nipponalba.scot, https://brid.gy/mastodon/@jk@social.nipponalba.scot I 2020-09-08 09:42:08.580631 Source: @jk@social.nipponalba.scot (Mastodon) @jk@social.nipponalba.scot, https://brid.gy/mastodon/@jk@social.nipponalba.scot I 2020-09-08 09:42:08.648537 Created by this poll: https://brid.gy/log?start_time=1599558066&key=agdicmlkLWd5cigLEghNYXN0b2RvbiIaQGprQHNvY2lhbC5uaXBwb25hbGJhLnNjb3QM>https://brid.gy/log?start_time=1599558066&Created by this poll: https://brid.gy/log?start_time=1599558066&key=agdicmlkLWd5cigLEghNYXN0b2RvbiIaQGprQHNvY2lhbC5uaXBwb25hbGJhLnNjb3QM/a> I 2020-09-08 09:42:08.649158 Starting Response like tag:social.nipponalba.scot,2013Starting Response like tag:social.nipponalba.scot,2013:9ywa42OcvvmGCeUDVw_favorited_by_9qEkWb1d76feMQOdRw https://social.nipponalba.scot/notice/9ywa42OcvvmGCeUDVw#favorited-by-9qEkWb1d76feMQOdRwhttps://social.nipponalba.scot/notice/9ywa42OcvvmGCeUDVw#favorited-by-9qEkWb1d76feMQOdRw I 2020-09-08 09:42:08.651036 Webmention from https://brid-gy.appspot.com/like/mastodon/@jk@social.nipponalba.scot/9ywa42OcvvmGCeUDVw/9qEkWb1d76feMQOdRw to https://jk.nipponalba.scot/note/5f574716/ I 2020-09-08 09:42:08.651310 Sending... I 2020-09-08 09:42:09.619104 Sent! {'request': 'POST https://webmention.io/jk.nipponalba.scot/webmention (with source=https://brid-gy.appspot.com/like/mastodon/@jk@social.nipponalba.scot/9ywa42OcvvmGCeUDVw/9qEkWb1d76feMQOdRw, target=https://jk.nipponalba.scot/note/5f574716/)';, 'http_status': 201, 'body': '{"status":"queued","summary":"Webmention was queued for processing","location":"https://webmention.io/jk.nipponalba.scot/webmention/o8JcupAF5h7gryInLVO2";,"source":"https://brid-gy.appspot.com/like/mastodon/@jk@social.nipponalba.scot/9ywa42OcvvmGCeUDVw/9qEkWb1d76feMQOdRw";,"target":"https://jk.nipponalba.scot/note/5f574716/";}'} I 2020-09-08 09:42:09.671017 Setting last_webmention_sent

This was before I updated from 2.0.7 to 2.1.0 - I saw the list of breaking changes in the change notes and so waited until I had the time to work through them before updating.

Since then I have successfully posted several times via brid.gy and have had various interactions on pleroma to the posts but no logs of any activity on my brid.gy page in the last 18 days.

feld · 2020-09-28T19:24:15Z

Evidently Mastodon's /api/v2/search endpoint handles a blank value for the limit query param ok, but Pleroma doesn't.

Is this believed to be the primary source of your issues with Pleroma?

snarfed · 2020-09-28T20:23:08Z

@feld i haven't done a thorough survey or test yet...but yes, this seems like the only thing blocking us right now.

having said that, you all deserve a lot of credit for how much feature parity you've achieved with the Mastodon API, at least the parts i'm using! this is obviously a trivial difference, and it will be easy to handle on my end, without even special casing Pleroma, by just omitting the limit param when i'm not providing a value. i'm happy to do that. just wanted to raise it on your radar in case you're intereseted.

feld · 2020-09-28T20:26:26Z

Our API is very strict now and invalid parameters throw an error instead of being ignored. I suspect that limit is not supposed to ever permit an empty value but Mastodon doesn't validate parameters as closely...

snarfed · 2020-09-28T20:32:39Z

sgtm! thanks for the info.

feld · 2020-09-28T20:36:28Z

Complementary issue over here now: https://git.pleroma.social/pleroma/pleroma/-/issues/2198

for snarfed/bridgy#977 , https://git.pleroma.social/pleroma/pleroma/-/issues/2198

snarfed · 2020-09-30T17:50:16Z

woo, fixed! https://brid.gy/mastodon/@jk@social.nipponalba.scot looks good now, sent a bunch of wms.

added in snarfed/granary@d05ad0f, for #977, https://git.pleroma.social/pleroma/pleroma/-/issues/2198

ScootRay · 2021-02-26T00:17:06Z

I noticed this error seems to have snuck back for Pleroma. It's reporting permission errors per my log:

https://brid.gy/log?start_time=1614298087&key=agdicmlkLWd5cmYLEg1QdWJsaXNoZWRQYWdlIj9odHRwczovL2Fsb25ndGhlcmF5LmNvbS9waG90b3Mvc3Vuc2V0LW9mZi1vZi1jYW1wb2JlbGxvLWlzbGFuZC8MCxIHUHVibGlzaBiAgIi52oaJCQw

I also looked at the logs for:

woo, fixed! https://brid.gy/mastodon/@jk@social.nipponalba.scot looks good now, sent a bunch of wms.

...and I see the same errors, whoops.

Is there anything I can do to help fix this?

Thanks,
Ray

snarfed · 2021-02-26T00:41:48Z

Thanks for the nudge! You're actually seeing a different error than this. I'll follow up on #1015.

snarfed added the new silo label Sep 27, 2020

snarfed added a commit to snarfed/granary that referenced this issue Sep 30, 2020

[mastodon] Omit empty limit param in search API call, for Pleroma

d05ad0f

for snarfed/bridgy#977 , https://git.pleroma.social/pleroma/pleroma/-/issues/2198

snarfed changed the title ~~official Pleroma support~~ Sep 30, 2020

snarfed closed this as completed Sep 30, 2020

snarfed added a commit that referenced this issue Sep 30, 2020

[mastodon] update tests to handle omitted limit param

e6ea890

added in snarfed/granary@d05ad0f, for #977, https://git.pleroma.social/pleroma/pleroma/-/issues/2198

snarfed mentioned this issue Oct 6, 2020

support Pleroma snarfed/bridgy-fed#12

Open

2 tasks

ScootRay mentioned this issue Feb 26, 2021

Pleroma (via Mastodon) publish: permission error #1015

Closed

snarfed mentioned this issue Mar 26, 2023

Misskey #1442

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix Pleroma search API incompatibility with empty `limit` param #977

fix Pleroma search API incompatibility with empty `limit` param #977

snarfed commented Sep 27, 2020

jk-na commented Sep 28, 2020 •

edited

Loading

snarfed commented Sep 28, 2020

jk-na commented Sep 28, 2020

feld commented Sep 28, 2020

snarfed commented Sep 28, 2020

feld commented Sep 28, 2020

snarfed commented Sep 28, 2020

feld commented Sep 28, 2020

snarfed commented Sep 30, 2020

ScootRay commented Feb 26, 2021

snarfed commented Feb 26, 2021

fix Pleroma search API incompatibility with empty limit param #977

fix Pleroma search API incompatibility with empty limit param #977

Comments

snarfed commented Sep 27, 2020

jk-na commented Sep 28, 2020 • edited Loading

snarfed commented Sep 28, 2020

jk-na commented Sep 28, 2020

feld commented Sep 28, 2020

snarfed commented Sep 28, 2020

feld commented Sep 28, 2020

snarfed commented Sep 28, 2020

feld commented Sep 28, 2020

snarfed commented Sep 30, 2020

ScootRay commented Feb 26, 2021

snarfed commented Feb 26, 2021

fix Pleroma search API incompatibility with empty `limit` param #977

fix Pleroma search API incompatibility with empty `limit` param #977

jk-na commented Sep 28, 2020 •

edited

Loading