Goodreads Developers discussion

It seems like a project that was started almost a decade ago and never really made a priority.

Some things are straight-up broken with no fix in sight (eg. friend update filters), some things work in an useless way (friend update "book" object doesn't have any image information?!), and it's all very messy xml responses and a non-standard oauth 1 spec.

And the main thing - Goodreads doesn't appear to even use this public api. The website isn't api-based, and they use a private, more functional api for their official app. So there's no reason for them to keep it updated; they aren't eating their own dog food.

Maybe this is a rant, but it's a question, too. Am I wrong in this? Are there plans to revitalize, modernize, and update this api?

I'm trying to create an Android app to provide users with a faster and more streamlined experience than the official app, but it doesn't seem possible to get there using the public api in its current state. Help us help you.

edit: Ok, they aren't allowed to distribute book cover images because of licensing agreements. Fair enough, although this alone makes it difficult to build an app. Every item on a list of books now needs to make an extra api call to another service...not good.

I Feeling you. I started building a wrapper for the API but it is all a mess. There is so much potential...

It's really frustrating. The official app is really "meh", even after the update. It's obvious that there's a metric shitton of legacy code at GR that they haven't dealt with and also an apparent lack of dev talent, or at least a failure to understand how to utilize their dev talent.

Like, they rebuilt their whole app, but it's online-first. And really slow. Even my girlfriend, who usually doesn't notice these things, doesn't like the app.

So my plan was to build an open-source app that has a fully async and modern UI that also caches as much as possible, and release it for free, maybe with a paid pro version.

But I'm just a little ways in and with the number of "wtf" moments I've had so far, I get the feeling I'm only gawking at the tip of an iceburg and trying to decide if I should keep sailing.

Pretty tempted to use the private api. Looks like it's the only way to actually get book cover urls for Friend Updates. Or...just give up and spend my time helping some other company that gives a shit. Sorry for negative tone, just frustrated.

is there a way to get at the private api?

A way? Yeah. An approved way from Goodreads that won't result in action from them? I don't know, but I would love to.

If there was indication from Goodreads that it's acceptable to use their private API - as is, with absolutely no guarantees of anything - then I'd be comfortable continuing work on an app. After all, it's the API that they use.

There's also the problem of the 1hit/second limitation of their public API. How exactly is one supposed to use their API to build something for the general public while guaranteeing that Goodreads won't rate limit them for their success?

All in all, it comes down to Goodread's business strategy, sadly. Is it important to them to get and keep people on their platform, even if it means losing access to the telemetry, ad revenue, and 3rd party purchase affiliate link revenue that they have with their official app? (even though all data must contain links back to GR, per the ToS)

Or would they welcome having unofficial apps that make users enjoy and recommend the platform, therefore recruiting others and driving more traffic to their site?

My suspicion is the former. If enough people used 3rd-party apps instead of their official app, they'd lose out on the stuff I mentioned, and it would make them look bad.

Regardless - would love some clarity. Would Goodreads allow an app on the Play Store that used their private API, since it's the only resource that's realistic to build an app against? Or will the public API get some attention?

yes it's a sad situation overall that they have so many fans willing to make it easier for their users to enjoy the goodreads experience, yet they have no interest. I'd understand if they were a very small team taking it as a side project.. But i honestly don't see why they are not investing in a better, more modern tech stack and api..

Hi David,
Are you by chance the person who mentioned using the API with Perl? Did you have success with that way back when? Do you have any examples? I decided I would like to develop a tool that would tell me what book series that I am reading has books I haven't read yet and then spit out the information. Author, Series Title. and Book # and Book Title of all unread books in the series.
A nice to-read list for me for series that I am reading. :)

Hey Charles,
I haven't done anything with Perl, no. My brief foray into the Goodreads API can be found abandoned here (Android app): https://github.com/davidwhitman/Tir

Pat wrote: "I Feeling you. I started building a wrapper for the API but it is all a mess. There is so much potential..."

Yup! I started that for Scala and gave up after a bit (well, I still kind of want to :)) because it is so slow and inconsistent. It's fun to use for small projects, but I couldn't imagine trying to make something very large, which I guess is the point from their perspective.

Alexis wrote: "Pat wrote: "I Feeling you. I started building a wrapper for the API but it is all a mess. There is so much potential..."

Yup! I started that for Scala and gave up after a bit (well, I still kind o..."

Same here. I can get the info I need via the GR API but I feel that my time is being wasted as there are no guarantees.

Now I am looking at the Google Books API.

David wrote: "Hey Charles,
I haven't done anything with Perl, no. My brief foray into the Goodreads API can be found abandoned here (Android app): https://github.com/davidwhitman/Tir"

Thanks David. I will take a look. I am hoping the task to read in the series of each books I have read and then check if more books exists in the series shouldn't be too hard, especially with all the nice abilities of perl (example hashes) to make flexible data structures...

I am sure other scripting languages have similar functions, but when you have a nice hammer - everything looks like a nail. :)
Just too bad I cant turn it into a nice standalone executable for others to use...

Hey David!

I'm pretty aligned with you on this.
I'll be building something, and will just have to work around the limitations. Throttled eventual functionality. Will start out with showing unread books from read authors with an ability to say no which will be stored in firebase. Think imdb recommendation feature.

On another note I was thinking of creating a proxy service for anyone to use, which could clean up mess, support oauth2 and introduce json for all endpoints.

On the 1 req/s limitation I thought every user/client could have their own apikey.

Was also thinking of opensourcing, alternatively a pro version.
Have you seen the SeriesGuide app?

Maybe a different route would be the google books api and limiting goodreads integration to user related data only (skipping author etc.)

Reach out if you're serious, I'd be happy to collaborate and build something of value.

http://www.johanilsson.com/

Hello, Johan,

> On the 1 req/s limitation I thought every user/client could have their own apikey.

I don't think that's possible.

From their "Using the API page":

> Some of our methods require a developer key, which you can get after you register. Most methods you can access just by passing your key, but some you'll need to use OAuth.

And the TOC:

>Not request any method more than once a second. Goodreads tracks all requests made by developers.

So, it's one per developer key, and you need to do an additional registration to get a developer key.

I suppose that you could try making every user register for a developer key. First ask them to sign in, then ask them for to register for a dev key. But I don't think that's feasible and GoodReads would probably have an issue with that.

I wish you the best of luck, but trying to develop against an intentionally crippled API isn't something that I'd like to pursue. The Google Books api is a really good idea, though.

I'd also like to voice my concern with how underwhelming the API is. I spent a good deal of time working on getting users to Auth with good reads so we could get more information about a user's books and lead them to find more books to read. GoodReads has a lot of potential here, but their restrictions and lack of core API functionality is making that difficult. I may need to wait until this is fixed. Until then, I'll have to check out the Google Books API.

A few things I'd love to see is: Better use of OAuth, finished endpoints (I can't even see what books a user has read without calling more than one endpoint per book, and it's still a hacky way), JSON responses, and less restrictive API (One call per second isn't something anyone can work with)

Hey Mark!
" and lead them to find more books to read"

This is what I was currently setting out to do.
I feel there should be one place that pulls in recommendations from different sites, but also what I would find extra useful would be finding unread books from selected authors. Currently working on that in a Nativescript phone app.

Is this something you're looking to undertake commercially or for fun?

I'm very much open to collaborations should there be any crossover in functionality between our ideas.

Agreed about the official app being of limited usefulness. I use it mainly just for scanning barcodes of books and I fallback on the web interface for everything else. Even in that regard it fails, as you can only scan one book at a time and the inbuilt barcode reader is quite frustrating to use.

Personally been investigating the API to help me manage my book collection, as it's not the easiest thing to rearrange hundreds of books between shelves.

Hey Daryl, I'm putting together an open sourced alternative app, which will provide these more powerful user features. Let me know if you want to pool resources together.

+1

Hi Johan, I've created an app with just the currently reading books and the ability to update them and offline first, it started with a broader scope but the API didn't help that much. I would really like for us to join our forces, although GR is trying to get a better app right now. The app I've wrote is called "Libellis for Goodreads" is on the Google play store if you want to check that out.

Please let me now what do you think!
Paul

Paul wrote: "Hi Johan, I've created an app with just the currently reading books and the ability to update them and offline first, it started with a broader scope but the API didn't help that much. I would real..."

Hi Pail, good work! I myself use a kindle device, which makes the whole journey of marking as currently reading and then read very smooth. My app is here https://play.google.com/store/apps/de... and because of it's features it's quite data and processing heavy. I'm sure we could figure out feature toggling and optimizations etc in the case of combining efforts into one app. Got a pretty straightforward approach of abstracting the ui.

Did you get around to implementing auth?

Happy to continue this conversation in private if you want. Just drop me a message.

Johan wrote: "Paul wrote: "Hi Johan, I've created an app with just the currently reading books and the ability to update them and offline first, it started with a broader scope but the API didn't help that much...."

Hi Joan, I did implement the auth. I send you a friend request to be able to talk via private message.

For those who are interested, and want to participate in continued discussion the thread is here: https://www.goodreads.com/topic/show/...

Hi, I was wondering if you guys ever find a way to get around the 1 reg/sec rate limit. Do you keep your app semi-private? Or do you have a caching layer? (afaik, we're allowed to store their data for 24 hours). I'd love to have any advices on what pitfalls to keep an eye on before starting.

(Some background: I'm planning to build a non-commercial app for facilitating the exchange of books with local friends. GR used to have a book swapping feature but they shut it down)

I would also love more clarification here from GR. I just wanted to build something simple and their API doesn't seem worth messing with.