Searchable NPR Book Concierge: 2008-2018

Last updated on May 12, 2020

In this post:

Intro: A reading list for the new year

NPR Love, Meet Dissertation Data

• Web-Scraping with Outwit Hub
• Tidying Data in RStudio

Snapshot of the Last 10 Years: Visualizations

• Total Number of Books by Year (2008-2018)
• Top 20 Tags Overall (2008-2018)
• Top 10 Tags by Year (2011-2018)

Conclusion

Tables

---

A reading list for the New Year

Episcopalians interrupting our fun to harshly remind us it’s still Christmas in 3, 2, 1

— Unvirtuous Abbey (@UnvirtuousAbbey) December 26, 2018

It is indeed still Christmastide! So I have a gift for everyone who has already blown through the books they found under the tree (or knows they will in this glorious lull between Christmas and New Year’s).

I web-scraped the NPR Book Concierge and Best Books of the Year pages from 2008 to 2018. And then I compiled them into a nifty spreadsheet that is searchable by tag and author or filterable by any category.

The spreadsheet contains 2323 books from the past 10 years of NPR’s “best of” lists. Search the tags to find new favorites in your most-read genres to take a chance on the “no-tag” books to discover something new.

For more on the process, keep reading the post. Though I understand if you go off and read the 2000+ awesome books from NPR’s Concierge instead :).

You can view the spreadsheet: 2008-2018 NPR Books or download a copy for yourself: Make a copy of 2008-2018 NPR Books. 

NPR Love, Meet Dissertation Data Skills

Since this year’s Book Concierge marks the 10th anniversary of NPR’s end-of-year recommendations, so I thought it’d be a fun side project to compile the lists from 2008 to 2018. This also gave me a chance to practice three of the digital humanities skills I’ve been using for the dissertation: web scraping plus tidy data and visualizations in R.

Web Scraping with OutWit Hub

First, I employed web-scraping using OutWit Hub. This is a method of harvesting data from a webpage by asking the scraping program to look for specific patterns in the HTML code behind a web page. 

Scraping the 2013 to 2018 pages was a breeze. The web app format NPR has used for the last five years is tagged well. For instance, <author> precedes the author’s name in the source code. The tags made it easy to identify common, unique patterns to plug into OutWit Hub’s desktop app.

Scraping the pages for 2008 to 2012, on the other hand, was a pain. (I’m still fairly new to web scraping and it stumped me.) The flow of the pages is inconsistent and I found it tricky to scrape just some links instead of all of them. Honestly, I ended up doing a lot of manual clean up, especially for 2008, 2009, and 2010. But okay. Worth it.

Tidying Data in RStudio

I also did some data tidying using the tidyverse and tidytext packages in RStudio. (I used Text Mining with R by Julia Silge and David Robinson as my bible for this work, as always.) Essentially, tidy data means that every instance of an observation has its own separate row in a table. For example, a book from the Concierge with two tags appears in two rows rather than one. (One for each tag.) To give a super simple example of untidy vs tidy data:

Figure 1: Untidy Data

Author	Book	Tags
Anne Leckie	Ancillary Justice	science-fiction-and-fantasy, it’s-all-geek-to-me

Figure 2: Tidy Data

Author	Book	Tag
Anne Leckie	Ancillary Justice	science-fiction-and-fantasy
Anne Leckie	Ancillary Justice	it’s-all-geek-to-me

Snapshot of the last 10 years

I mainly tidied the data so I could show off some visualizations. 🙂 Voila –  total counts for year and tag plus most used tags by year.

Total number of books per year

Graph: Total books per year

Interestingly, a bit of a dip in the number of books in the concierge this year (2018). But otherwise the count keeps going steadily up. We should see upwards of 320 in 2019 at least.

Top 20 Tags (2008-2018)

Over the last 10 years, the “best of” lists utilized 74 distinct tags. This gets visually messy, so I’ve opted to show just the top 25 here. I removed the “no-tag” label, which applied to all books in the 2008, 2009, and 2010 “best of” lists.

Please also note there’s some overlap. “Staff-picks” and “npr-staff-pics,” for example, are essentially the same thing. Overall, 760 of the 2323 total book (37.72%) carry one of the “staff-pick” tags.

Graph: Top 25 Tags (2008-2018)

Top 10 Tags by Year (2011-2018)

The most popular tags have shifted from year to year, with some tags fading out altogether over time. The facets below show the most popular tags per year from 2011 to 2018 (since no tags are available for 2008-2010). My apologies for the messy visualizations. I’m still working on how to sort the darn things properly…

Conclusion

This was a fun project for my winter break and definitely something I’ll use going forward. My next step is to start tracking which books I’ve read and locate the ones I most want to read in my local library.

Suggestions welcome for other improvements. Otherwise, happy reading!

---

Tables

Table 1: Total books per year

Year	Total Books
2008	108
2009	106
2010	128
2011	126
2012	135
2013	205
2014	253
2015	271
2016	309
2017	374
2017	219

Table 2: Top 25 Tags (2008-2018)

Tag	Total Books
staff-picks	633
nonfiction	395
the-dark-side	377
tales-from-around-the-world	368
realistic-fiction	367
family-matters	363
seriously-great-writing	360
book-club-ideas	314
for-history-lovers	293
eye-opening-reads	282
identity-and-culture	272
ladies-first	261
science-fiction-and-fantasy	246
rather-long	184
love-stories	172
biography-and-memoir	166
funny-stuff	150
kids-books	147
historical-fiction	146
mysteries-and-thrillers	143
book-club-ideas	138
for-art-lovers	137
npr-staff-picks	127
poetry	126
family-matters	125

Table 3: Top 10 Tags by Year

Year	Tag	Count
2011	nonfiction	26
2011	literary-fiction	16
2011	arts-and-entertainment	15
2011	historical-fiction	11
2011	food-and-wine	10
2011	childrens-books	9
2011	fiction	9
2011	young-adult	8
2011	poetry	6
2011	mysteries-thrillers-and-crimes	4
2011	science-fiction-and-fantasy	4
2012	nonfiction	38
2012	fiction	15
2012	mysteries-thrillers-and-crimes	12
2012	science-fiction-and-fantasy	11
2012	historical-fiction	9
2012	poetry	9
2012	arts-and-entertainment	8
2012	literary-fiction	8
2012	childrens-books	5
2012	comics-and-graphic-novels	5
2013	for-history-lovers	60
2013	seriously-great-writing	52
2013	the-dark-side	52
2013	family-matters	51
2013	book-club-ideas	49
2013	realistic-fiction	42
2013	tales-from-around-the-world	36
2013	npr-staff-picks	30
2013	rather-long	30
2013	eye-opening-reads	29
2013	funny-stuff	29
2013	npr-staff-picks	29
2014	npr-staff-picks	98
2014	realistic-fiction	76
2014	tales-from-around-the-world	71
2014	the-dark-side	67
2014	science-and-society	61
2014	seriously-great-writing	59
2014	family-matters	47
2014	for-history-lovers	44
2014	eye-opening-reads	42
2014	science-fiction-and-fantasy	39
2015	staff-picks	122
2015	seriously-great-writing	72
2015	realistic-fiction	70
2015	nonfiction	69
2015	tales-from-around-the-world	65
2015	the-dark-side	60
2015	family-matters	58
2015	science-fiction-and-fantasy	50
2015	book-club-ideas	46
2015	eye-opening-reads	39
2016	staff-picks	148
2016	nonfiction	95
2016	the-dark-side	67
2016	tales-from-around-the-world	66
2016	seriously-great-writing	65
2016	family-matters	64
2016	ladies-first	64
2016	identity-and-culture	58
2016	realistic-fiction	58
2016	book-club-ideas	55
2017	staff-picks	206
2017	nonfiction	116
2017	realistic-fiction	92
2017	identity-and-culture	81
2017	tales-from-around-the-world	75
2017	ladies-first	74
2017	the-dark-side	71
2017	family-matters	64
2017	book-club-ideas	63
2017	for-history-lovers	57
2018	staff-picks	157
2018	nonfiction	115
2018	identity-and-culture	103
2018	the-states-were-in	97
2018	ladies-first	87
2018	family-matters	79
2018	eye-opening-reads	75
2018	book-club-ideas	69
2018	realistic-fiction	61
2018	the-dark-side	60