Tuesday, July 8, 2014

Finding the Best Book Quotes: Power Ranking Goodreads

My friend Jordan Ellenberg wrote an article for the Wall Street Journal recently describing a metric to roughly measure the least read books, which he calls the Hawking Index (HI). As he describes it, take the page numbers of a book's five top highlights, average them, and divide by the number of pages in the whole book.

On a discussion thread on Facebook, this led to a proposal from me for measuring the general quality of a quote. Assume a user has some level of discrimination \(D\); the higher the value of \(D\), the more likely they are to quote a passage. Now assume each passage has some measure of quality \(Q\); the higher the value of \(Q\), the more likely a passage is to be quoted. Let's try a classic Bradley-Terry-Luce model - if a user with discrimination \(D\) quotes at least one passage from a particular work, the probability \(p\) that they'll quote a given passage with quality \(Q\) from that same work is roughly \[ p = \frac{Q}{Q+D}.\]
Nice, but where can we find data to actually test this theory? It turns out an excellent source is Goodreads. Users are numbered sequentially starting with Goodreads founder Otis Chandler's ID of 1; a user's associated quote page has a simple URL structure. Furthermore, every user's quote list has an associated RSS/Atom URL that provides an easy to parse XML file.

The steps:
  1. Write a simple Go scraper to uses a goroutine to quickly and sequentially get user quote pages, grab the count of quotes shown on that page (truncated at 30), then grab the RSS/Atom URL. This is able to process about 2500/users minutes over a slow internet connection.
  2. Write a simple Ruby program to fetch quote XML files for every user that has at least one quote.
  3. Write a simple Ruby program to fetch the author URL and work URL for every quote cited by at least one user.
  4. Load all this data into a PostgreSQL database.
  5. Write a simple R program to pull the data from the database and fit our model. We assign a value for the outcome "quoted" of 1 if the user quoted a passage; we assign "quoted" the value 0 if the user quote a passage from that work, but not that particular passage. Both user "discrimination" and quote "quality" are treated as factors and modeled as random effects; the R package "lmer" was used to fit the model.
If there is interest, I can make all of this code publicly available through my GitHub account. None of it is proprietary.

As a test I grabbed the data from 333760 users, 10724 had at least one quote, there were 83160 total quotes and 32621 unique quotes. The model took about 3.5 minutes to fit using an optimized BLAS library and 4 hyperthreaded CPU cores.

The top 10 quotes on Goodreads, ranked by quality:
  1. “No one can make you feel inferior without your consent.”
    Eleanor Roosevelt, This is My Story
  2. “Outside of a dog, a book is man's best friend. Inside of a dog it's too dark to read.”
    Groucho Marx, The Essential Groucho: Writings For By And About Groucho Marx
  3. “Twenty years from now you will be more disappointed by the things that you didn't do than by the ones you did do. So throw off the bowlines. Sail away from the safe harbor. Catch the trade winds in your sails. Explore. Dream. Discover.”
    H. Jackson Brown Jr., P.S. I Love You
  4. “I am so clever that sometimes I don't understand a single word of what I am saying.”
    Oscar Wilde, The Happy Prince and Other Stories
  5. “Insanity is doing the same thing, over and over again, but expecting different results.”
    Narcotics Anonymous, Narcotics Anonymous
  6. “He has achieved success who has lived well, laughed often, and loved much;Who has enjoyed the trust of pure women, the respect of intelligent men and the love of little children;Who has filled his niche and accomplished his task;Who has never lacked appreciation of Earth's beauty or failed to express it;Who has left the world better than he found it,Whether an improved poppy, a perfect poem, or a rescued soul;Who has always looked for the best in others and given them the best he had;Whose life was an inspiration;Whose memory a benediction.”
    Bessie Anderson Stanley, More Heart Throbs Volume Two in Prose and Verse Dear to the American People And by them contributed as a Supplement to the original $10,000 Prize Book HEART THROBS
  7. “The trouble with having an open mind, of course, is that people will insist on coming along and trying to put things in it.”
    Terry Pratchett, Diggers
  8. “Love all, trust a few, do wrong to none.”
    William Shakespeare, All's Well That Ends Well
  9. “The Paradoxical Commandments
    People are illogical, unreasonable, and self-centered.
    Love them anyway.
    If you do good, people will accuse you of selfish ulterior motives.
    Do good anyway.
    If you are successful, you will win false friends and true enemies.
    Succeed anyway.
    The good you do today will be forgotten tomorrow.
    Do good anyway.
    Honesty and frankness make you vulnerable.
    Be honest and frank anyway.
    The biggest men and women with the biggest ideas can be shot down by the smallest men and women with the smallest minds.
    Think big anyway.
    People favor underdogs but follow only top dogs.
    Fight for a few underdogs anyway.
    What you spend years building may be destroyed overnight.
    Build anyway.
    People really need help but may attack you if you do help them.
    Help people anyway.
    Give the world the best you have and you'll get kicked in the teeth.
    Give the world the best you have anyway.”
    Kent M. Keith, The Silent Revolution: Dynamic Leadership in the Student Council
  10. “She is too fond of books, and it has turned her brain.”
    Louisa May Alcott, Work: A Story of Experience

6 comments:

  1. as you are using Pairwise comparison, i think the order of comparison would effect the result. there is a famous Condorcet Paradox..

    it would be great, if you have the code published on github :)

    ReplyDelete
  2. Awesome. Also, post a psql dump of the DB to github so we don't have to abuse goodreads' servers.

    ReplyDelete
  3. Many of us who went last year made the same trip, so it was good to see ... but it's our job to have as Great in number, fits as we can in a week. .... Next time you are in Guatemala, I would like to invite you to have lunch with me at my restaurant (www.casayurritaeventos

    ReplyDelete
  4. This time, we tend to got the subsequent problem clue: "Sesame Street" store owner that conjointly referred to as "Sesame Street" store owner lexicon. First, we tend to gonna hunt for additional hints to the "Sesame Street" store owner problem. Then we are going to collect all the desired info and for resolution "Sesame Street" store owner problem . within the final, we tend to get all the attainable answers for the this problem definition.

    ReplyDelete
  5. “No one can make you feel inferior without your consent.” Eleanor Roosevelt, This is My Story -- I agree! So in case someone likes to make you horrible, fight bac and prove that you are more than that! Thank you for consolidating these inspirational sayings.

    ReplyDelete
  6. Did you mean "the higher the value of D, the _less_ likely they are to quote a passage"? Because the formula gives p = 1 for D=0 and p->0 for D-> infinity.

    ReplyDelete