Wednesday, June 7, 2017

Lyric Intelligence In Popular Music: A Ten Year Analysis

Lyric Intelligence In Popular Music: A Ten Year Analysis

Popular music lyrics are dumb. No really, I’m not just saying that. As easy as it is to mock the quality of lyrics today, there’s some real science behind looking at how dumb they truly are.
That’s why I set out to answer the big questions. Which genre is the most sophisticated? (Prepare to be disappointed.) Which artists are the dumbest? (Prepare to be surprised.) And, can any hit songs be comfortably read by a 1st grader? (Yes, they can.)
How exactly did I go about this?
I turned to the Readability Score. It uses writing analysis tools like the Flesch-Kincaid grade index and many others to create an average of the US reading level of a piece of text. I plugged in song lyrics (punctuation added by me, since most songs lack it altogether) and out of the machine popped out average grade level, word count, and other very interesting metrics.
All told, I analyzed 225 songs in 4 different datasets, resulting in 2,000+ individual data points. How’d I choose them? If they spent at least a few weeks (3+) at #1 on the Billboard charts for Pop, Country, Rock, and R&B/Hip-Hop for any given year, they made the list.
While the results are certainly enlightening, it’s important to note that this data doesn’t touch on the meaning of a song, the metaphors, how the words connect with the artist’s personal story, etc. to create deeper meaning. These numbers are fun and interesting, so just enjoy them.
What did the data tell us?

Is We Really Getting Dumber?


Well, this research is, if nothing else, great news for third graders. They may have a long way to go in terms of unlocking the deeper meanings of great literature, but hit music lyrics are right in their zone.
Did you miss that? Yes, I said third graders. Because only 2005 and 2006 would have given a reasonably competent third grader a tough time. Sure, we know hit music lyrics aren’t the peak of sophistication, but who knew the bar was this low?
Well, the bar is actually getting lower. These averages have really been on the decline over the last 10 years. And keep in mind that we’re looking at all the data from Pop, Rock, R&B/Hip Hop, and Country combined to show the overall trend.
So it’s clear we’re on the way down, but if you’re wondering “Was it men or women dragging the music down?” you’re not the only one. I was just as curious to find out.

 Who Wins the Battle of the Sexes?

The short answer is that everybody is getting dumber, it’s not really a gender thing.  Overall, my data showed that men talk more (higher word count), but women have slightly more intelligent things to say (higher average grade level).
Let’s take a look the average grade level over time to see how men and women fare.

It looks like hit music in general has settled on some formulas: Men should stick to roughly a 3rd grade reading level and women shouldn’t sing rock songs at all (more about that later).
What does it all mean exactly? Well, women seem to be a bit smarter than men, except for when they’re not (i.e., 2008-2009). Ultimately, the genre and the artist matter much more than the gender.
Granted, it’s not exactly a team effort. Certain artists do more than their fair share to bring their gender and genre group down or up through the years. So now it’s time to break down this data a bit more by genre and see who really sets the standard for dumb.

A Deeper Look into Genres


Does Country really have the “smartest” Lyrics?
Yes, yes, it does. In fact, if you average out all the hit songs in each category for the last 10 years, you get:
Average US Reading Level by Grade:
Country: 3.3
Pop: 2.9 (tie)
Rock: 2.9 (tie)
R&B/Hip Hop: 2.6
There are a lot of reasons for this. Remember that I mentioned that word length plays a role? Well, Country is the only genre generally devoid of words like “oh” or “yeah” repeated 20 times in a row. Sorry everyone else, but if you say it in the song, it’s counted as a “lyric.”
But it’s also about the syllables. Country music is full of words like Hallelujah, cigarettes, hillbilly, and tacklebox. Add to that long place names like Cincinnati, Louisville, Mississippi, and Louisiana, and Country has a serious advantage over the competition.
Unfortunately for Pop and R&B/Hip-Hop, places like L.A. and New York just don’t score that many points. But take a song like Dani California, and you’ll see that throwing in the word “California” more than a dozen times can make a real difference.
In 2007, Rock and R&B/Hip-Hop both plunged with the help of songs like “Buy U a Drank” by T-Pain (which just made it above a 1st grade reading level) and “I Don’t Wanna Stop” by Ozzy Osbourne (a more respectable 1.6 average grade level).
Then, in 2009, it was Country and Pop’s turn, when they each hit their own record lows with the help of songs like “Boom Boom Pow” by the Black Eyed Peas (2.0 grade level) and “Then” by Brad Paisley (which wasn’t much higher at a 2.2 grade level).
Was it the housing boom and bust that got people in the mood for listening to the simpler songs in life? Hard to say, but what’s for sure is that the people have spoken. No matter what the year and the economic situation, they do not want to listen to anything a third grader would find challenging.
Looking at the overall trend, it seems things have leveled off since 2010, with all four major genres resting comfortably into the 2nd-3rd grade range and R&B/Hip Hop ranking last in 5 of the past 10 years. Sorry 1st graders, but you’ll have to settle for R&B and Hip Hop from 2007.
Still, besides those grade level averages, how do elements like word count play into the picture?

Interestingly enough, there’s a definite correlation here between the average length of song and the average grade level. Problem is, it’s an inverse relationship — shorter songs actually have a higher grade level.
Shockingly, R&B/Hip Hop and Pop seem to be talking a lot and not saying much.
Country and Rock, on the other hand, certainly seem to have a winning formula: keep it short, while not skimping on word or grammar complexity, and you have nicely aged song. It’s surprising what you can fit into less than 300 words.
What’s clear is that over time, not a lot has changed. Aside from normal fluctuations, all the genres seem to have a standard level they stick to. Frankly, having listened to my fair share of radio for the past 10 years, that’s not exactly surprising.

Artist VS. Artist: How Low Can They Go?

Hint: Very Low.
I went through each category and picked 7 of the top artists based on their number of hit songs and how long those songs stayed at #1. So without further ado, here are the facts about the artists whose voices we hear most often in each genre:

This is probably a good time to reiterate that these are the numbers, plain and simple, and that all BeyoncĂ© partisans should address their complaints to the people at the Readability Score. We can’t hide the facts: of these 7 top R&B and Hip Hop artists, she has the second smallest average word count and the least sophisticated lyrics.
Who’s the standout? None other than Slim Shady himself, with Nicki Minaj and Macklemore following up in the number 2 and 3 spots. Is Kanye West talking a lot and not saying anything? Yes. Should you be surprised?
Wait, I’mma Let You Finish. No.

The Country data here seems to be telling us something. The more devoted teenage girls are to an artist, the more poorly these artists score. Okay, that wasn’t exactly fair, but Taylor Swift, Rascal Flatts, and Florida Georgia Line are bottom of the pack here in terms of average grade level at 3.4, 3.2 and 2.93.
The queen of Country, however, is definitely Carrie Underwood with a grade level of 3.72. Blake Shelton (3.63) and Brad Paisley (3.54) are close behind. If there’s one trend clearly at play here, it’s that (again) longer lyrics don’t exactly translate into more sophisticated lyrics. But, perhaps Rock has something to say about that.

Yes, Rock has plenty to say about the “longer lyrics = less sophisticated lyrics” trend. Not just any Rock either, we’re talking about Nickelback. Yes, the Canadian rock band so many love to hate manages to get the longest AND most sophisticated lyrics in the Rock category.
Just as surprising, Linkin Park came in with the shortest lyrics and the second most sophisticated. Who knew? Still, on the whole, major Rock artists and groups stay pretty firmly in the 2nd and occasionally 3rd grade. I guess guitars can compensate for just about anything.
If there’s a single stunning fact about this genre, it still has to be that not a single female singer made it onto the Rock list. That means not a single lady stayed at the top of the charts for four or more weeks during the entire last 10 years. Why that might be frankly deserves its own post.
But how does this all compare to what’s maybe the most generic, the most ubiquitous, and the most popular category of all? Ke$ha fans, prepare yourselves…

As promised, Ke$ha does manage to score the worst of any major artist I looked more closely at. Not just a little worse either, although Lady Gaga makes a valiant effort to keep up (keep down?).
Unlike the other genres, the Pop stars who stand out are a bit less surprising with the gold, silver, and bronze going to Mariah Carey, Adele, and Justin Timberlake, respectively.
Pop actually holds its own pretty well with Mariah Carey scoring higher than anyone in the other three categories. Bravo Mariah, bravo!
But how do artists and their songs compare across genres?

Top Songs: The Good, The Bad, and The Very Ugly

Remember how I mentioned that Country has done very well? Well, here’s a bit of insight into why. Fully 5 of the top 10 smartest songs in this study are country songs. Only a single song (thanks Rihanna) made it from R&B/Hip Hop.

This of course brings us to the shame list, the dumbest 10 songs of the last 10 years. Here, Country has completely vanished, replaced by a fairly even representation of Pop, Rock, and R&B/Hip Hop.
Somehow, Three Days Grace and Maroon 5 managed to score on both charts. The former scoring the dumbest song in the entire study alongside the 3rd smartest.

Showing My Homework

For anyone who’s a bit more curious, here’s the full list of the four categories ranked by average grade level.

Top R&B/Hip-Hop Songs Ranked by Grade Level

Average Grade Level Title Artist Year Weeks at #1 Word Count Characters per Word Syllables per Word
4.8 Diamonds Rihanna 2012 11 382 4 1.4
4.1 Thrift Shop Macklemore & Ryan Lewis Featuring Wanz 2013 14 725 3.9 1.3
3.9 Irreplaceable Beyoncé 2006 9 549 3.5 1.2
3.9 Slow Down Bobby Valentino 2005 4 452 3.8 1.2
3.8 Moment 4 Life Nicki Minaj Featuring Drake 2011 5 648 3.7 1.2
3.7 The Monster Eminem Featuring Rihanna 2013 13 710 3.7 1.2
3.7 We Belong Together Mariah Carey 2005 14 418 3.5 1.2
3.4 Mercy Kanye West, Big Sean, Pusha T, 2 Chainz 2012 4 968 3.9 1.2
3.4 I Remember Keyshia Cole 2008 7 319 3.7 1.2
3.3 No One Alicia Keys 2007 17 314 3.2 1.2
3.3 Let Me Love You Mario 2005 11 477 3.7 1.2
3.2 Best I Ever Had Drake 2009 7 811 3.5 1.2
3.2 Hold On, We’re Going Home Drake Featuring Majid Jordan 2013 5 343 3.5 1.2
3.2 Pretty Wings Maxwell 2009 14 267 3.9 1.2
3.2 Lotus Flower Bomb Wale Featuring Miguel 2012 5 589 3.6 1.2
3.1 Be Without You Mary J. Blige 2006 15 575 3.4 1.2
3 Anaconda Nicki Minaj 2014 6 641 3.4 1.2
2.9 Deuces Chris Brown Featuring Tyga & Kevin McCall 2010 9 596 3.4 1.2
2.8 I’m On One DJ Khaled Featuring Drake, Rick Ross & Lil Wayne 2011 10 722 3.4 1.2
2.8 Motivation Kelly Rowland Featuring Lil Wayne 2011 6 369 3.5 1.2
2.8 Heaven Sent Keyshia Cole 2008 9 495 3.6 1.2
2.7 Look At Me Now Chris Brown Featuring Lil Wayne & Busta Rhymes 2011 8 906 3.4 1.2
2.7 Everything To Me Monica 2010 7 201 3.5 1.2
2.7 Can’t Be Friends Trey Songz 2010 11 358 3.2 1.1
2.7 Soul Survivor Young Jeezy Featuring Akon 2005 4 792 3.4 1.2
2.6 She Will Lil Wayne Featuring Drake 2011 4 794 3.5 1.2
2.6 Blurred Lines Robin Thicke Featuring T.I. + Pharrell 2013 16 674 3.3 1.2
2.5 Candy Shop 50 Cent Featuring Olivia 2005 4 619 3.4 1.2
2.4 Ni**as in Paris Jay Z Kanye West 2011 7 576 3.5 1.2
2.4 Can’t Hold Us Macklemore & Ryan Lewis Featuring Ray Dalton 2013 7 758 3.4 1.2
2.4 Happy Pharrell Williams 2014 12 544 3.5 1.2
2.4 Love In This Club Usher Featuring Young Jeezy 2008 4 613 3.3 1.1
2.4 It’s Goin’ Down Yung Joc 2006 8 589 3.5 1.2
2.2 Gold Digger Kanye West Featuring Jamie Foxx 2005 4 774 3.5 1.1
2.2 Lollipop Lil Wayne Featuring Static Major 2008 6 658 3.4 1.2
2.2 There Goes My Baby Usher 2010 4 456 3.5 1.2
2.1 Un-Thinkable (I’m Ready) Alicia Keys 2010 12 384 3.3 1.2
2.1 Black Widow Iggy Azalea Featuring Rita Ora 2014 5 551 3.4 1.2
2.1 Climax Usher 2012 10 326 3.4 1.2
2 Blame It Jamie Foxx Featuring T-Pain 2009 14 803 3.3 1.2
2 Adorn Miguel 2012 4 265 3.5 1.2
1.9 When I See U Fantasia 2007 8 380 3.3 1.1
1.9 Fancy Iggy Azalea Featuring Charli XCX 2014 13 535 3.4 1.2
1.8 Love On Top Beyonce 2012 7 599 3.3 1.2
1.8 What You Know T.I. 2006 6 870 3.5 1.2
1.7 No Lie 2 Chainz Featuring Drake 2012 5 783 3.3 1.1
1.6 Single Ladies (Put A Ring On It) Beyonce 2008 12 687 3 1.1
1.6 Say Goodbye Chris Brown 2006 4 793 3.2 1.2
1.6 Bed J. Holiday 2007 5 597 3.3 1.2
1.6 Need U Bad Jazmine Sullivan 2008 4 479 3 1.1
1.2 Lost Without U Robin Thicke 2007 11 396 3.3 1.2
1.2 Buy U A Drank (Shawty Snappin’) T-Pain Featuring Yung Joc 2007 8 545 3.4 1.1
1 It Kills Me Melanie Fiona 2010 9 316 3.3 1.1
In the R&B/Hip Hop category, Rihanna’s Diamonds may seem like a surprising choice for sophistication, but throwing those rocks around got her average word length up to a full 4 characters. Macklemore’s high score, on the other hand, seems to speak for itself. Respect the Jammies is all I’m saying.
Then there’s Robin Thicke and Melanie Fiona… What can we say about the subtle metaphors and other intricacies which surely lie behind lines like “Tell me how you love my body?” Not a lot, really. Like Macklemore, they speak fairly well for themselves.

Top Country Songs Ranked by Grade Level

Average Grade Level Song Title Artist Year Weeks at #1 Word Count Syllables per Word Characters per Word
5.8 All About Tonight Blake Shelton 2010 3 311 1.3 3.9
4.9 That’s What I Love About Sunday Craig Morgan 2005 4 254 1.2 3.7
4.9 We Were Us Keith Urban And Miranda Lambert 2013 3 283 1.3 3.9
4.7 Before He Cheats Carrie Underwood 2006 5 304 1.2 3.8
4.6 Good Directions Billy Currington 2007 3 303 1.2 3.8
4.6 The House That Built Me Miranda Lambert 2010 4 310 1.2 3.7
4.5 We Are Never Ever Getting Back Together Taylor Swift 2012 9 376 1.3 3.8
4.4 The World Brad Paisley 2006 3 248 1.3 3.6
4.4 Summertime Kenny Chesney 2006 5 294 1.2 3.6
4.1 Beer In Mexico Kenny Chesney 2007 3 242 1.3 3.7
4 I’m Still A Guy Brad Paisley 2008 3 364 1.1 3.5
4 Making Memories Of Us Kieth Urban 2005 5 247 1.2 3.5
3.9 So Small Carrie Underwood 2007 3 274 1.2 3.9
3.9 Take Me There Rascal Flatts 2007 3 256 1.2 4
3.7 She’s Everything Brad Paisley 2007 3 358 1.3 3.5
3.7 Consider Me Gone Reba 2010 4 303 1.2 3.4
3.7 If You’re Going Through Hell (Before The Devil Even Knows) Rodney Atkins 2006 4 362 1.1 3.6
3.6 Sure Be Cool If You Did Blake Shelton 2013 5 371 1.2 3.6
3.6 Something In The Water Carrie Underwood 2014 3 340 1.2 3.6
3.6 Watching You Rodney Atkins 2007 4 398 1.2 3.5
3.5 Wasted Carrie Underwood 2007 3 343 1.2 3.6
3.5 Better Life Kieth Urban 2005 6 325 1.2 3.4
3.5 Shutgun Rider Tim McGraw 2014 3 277 1.3 3.7
3.4 Letter To Me Brad Paisley 2008 4 450 1.2 3.4
3.3 Don’t Blink Kenny Chesney 2007 4 383 1.1 3.8
3.2 Stay Florida Georgia Line 2013 6 410 1.2 3.4
3.2 Somewhere With You Kenny Chesney 2011 3 374 1.2 3.4
3.2 Drink A Beer Luke Bryan 2014 5 176 1.2 3.6
3 Don’t You Wanna Stay Jason Aldean With Kelly Clarkson 2011 3 222 1.2 3.6
3 Why Don’t We Just Dance Josh Turner 2010 4 230 1.1 3.5
3 Living In Fast Forward Kenny Chesney 2006 3 235 1.2 3.6
3 Play It Again Luke Bryan 2014 9 436 1.2 3.5
2.9 Honey Bee Blake Shelton 2011 4 355 1.2 3.3
2.9 Jesus, Take The Wheel Carrie Underwood 2006 6 307 1.1 3.4
2.9 Cruise Florida Georgia Line 2013 24 421 1.2 3.7
2.9 Bless The Broken Road Rascal Flatts 2005 5 239 1.2 3.7
2.8 What Hurts The Most Rascal Flatts 2006 4 280 1.2 3.6
2.7 This Is How We Roll Florida Georgia Line Featuring Luke Bryan 2014 6 383 1.2 3.6
2.7 Burnin’ It Down Jason Aldean 2014 14 436 1.2 3.7
2.7 That’s My Kind Of Night Luke Bryan 2013 12 390 1.1 3.6
2.6 Never Wanted Nothing More Kenny Chesney 2007 5 352 1.2 3.3
2.6 Felt Good On My Lips Tim McGraw 2011 3 326 1.2 3.4
2.5 Big Green Tractor Jason Aldean 2009 4 368 1.1 3.3
2.5 Need You Now Lady Antebellum 2009 5 237 1.2 3.3
2.5 Our Song Taylor Swift 2008 6 386 1.2 3.6
2.5 You’re Gonna Miss This Trace Adkins 2008 3 317 1.2 3.5
2.5 Keep Me In Mind Zac Brown Band 2012 4 287 1.2 3.4
2.3 It Won’t Be Like This For Long Darius Rucker 2009 3 317 1.1 3.4
2.2 God Gave Me You Blake Shelton 2011 3 274 1.1 3.3
2.2 Then Brad Paisley 2009 3 285 1.1 3.5
2.1 The Man I Want To Be Chris Young 2010 3 266 1.1 3.2
1.5 As Good As I Once Was Toby Keith 2005 6 380 1.1 3.2
Country scored very well overall; what kind of lyrics could have made that happen?
Let’s just say they got some help from feel good pills and red Gatorade. I already mentioned the long words which brought country to the champion’s table (tackleboxes and cigarettes anyone?), but longer sentences also played a part (thanks Keith Urban).
Of course, even for Country, it wasn’t all roses and bourbon.
Toby Keith and Chris Young did the genre no favors with their pleads to God and their baby. These lyrics scored low with a combination of short phrases and clippy writing. If only they could have mentioned Mississippi more…

Top Rock Songs Ranked by Grade Level

Average Grade Level Title Artist Year Weeks at #1 Word Count Characters per Word Syllables per Word
5.5 Dani California Red Hot Chili Peppers 2006 12 296 4.2 1.5
5.2 Animal I Have Become Three Days Grace 2006 7 258 3.9 1.4
4.5 Be Yourself Audioslave 2005 7 194 3.9 1.3
4.5 Inside The Fire Disturbed 2008 14 293 4 1.3
4.3 The Pot Tool 2006 4 320 4.1 1.3
4.2 Check My Brain Alice In Chains 2009 8 148 4 1.4
4.2 Isolation Alter Bridge 2011 7 218 4.1 1.4
4.2 Another Way To Die Disturbed 2010 8 321 3.9 1.3
4.2 Something In Your Mouth Nickelback 2009 4 470 4.1 1.3
4 Say You’ll Haunt Me Stone Sour 2010 8 369 3.8 1.3
3.9 Your Decision Alice In Chains 2010 8 162 3.7 1.2
3.7 Boulevard Of Broken Dreams Green Day 2005 14 289 3.5 1.2
3.5 New Divide Linkin Park 2009 8 250 4 1.2
3.4 Psycho Puddle Of Mudd 2008 9 338 3.6 1.3
3.4 Remedy Seether 2005 5 355 3.5 1.1
3.4 Second Chance Shinedown 2009 10 250 3.8 1.3
3.3 Speak Godsmack 2006 6 181 3.7 1.1
3.2 Lifeline Papa Roach 2009 6 283 3.7 1.3
3.2 Through Glass Stone Sour 2006 7 520 3.8 1.2
3.1 Save Me Shinedown 2005 12 199 3.8 1.3
3 Best Of You Foo Fighters 2005 4 338 3.7 1.2
3 Break Three Days Grace 2010 11 187 3.9 1.3
2.9 What I’ve Done Linkin Park 2007 8 158 3.5 1.2
2.9 Animals Nickelback 2006 6 480 3.6 1.2
2.9 Chalk Outline Three Days Grace 2012 13 244 3.6 1.2
2.8 Walk Foo Fighters 2011 4 305 3.7 1.3
2.8 Photograph Nickelback 2005 7 489 3.5 1.2
2.8 Country Song Seether 2011 10 411 3.5 1.1
2.7 The Pretender Foo Fighters 2007 6 499 3.4 1.2
2.7 Cryin’ Like A Bitch! Godsmack 2010 5 258 3.7 1.2
2.7 Painkiller Three Days Grace 2014 4 249 3.5 1.2
2.6 Face To The Floor Chevelle 2011 12 216 3.7 1.2
2.6 Fake It Seether 2008 7 339 3.6 1.2
2.5 Shepherd Of Fire Avenged Sevenfold 2014 7 273 3.6 1.2
2.4 Rope Foo Fighters 2011 5 264 3.4 1.1
2.3 Trenches Pop Evil 2013 4 287 3.5 1.1
2.3 Words As Weapons Seether 2014 5 239 3.5 1.1
2.3 Been Away Too Long Soundgarden 2013 5 239 3.6 1.2
2.2 Something From Nothing Foo Fighters 2014 13 306 3.5 1.2
2.2 Live To Rise Soundgarden 2012 6 225 3.5 1.1
2.1 Pain Three Days Grace 2007 13 332 3.6 1.2
2 Hail To The King Avenged Sevenfold 2013 10 246 3.8 1.1
1.9 Breath Breaking Benjamin 2007 7 234 3.4 1.1
1.8 Never Too Late Three Days Grace 2007 7 297 3.3 1.2
1.7 The Day That Never Comes Metallica 2008 7 182 3.8 1.1
1.7 Heaven Knows The Pretty Reckless 2014 5 338 3.6 1.2
1.6 I Don’t Wanna Stop Ozzy Osbourne 2007 5 406 3 1.2
1.6 Bully Shinedown 2012 12 369 3.3 1.2
1.6 World So Cold Three Days Grace 2010 5 265 3.4 1.2
1.4 Unity Shinedown 2012 4 372 3.3 1.1
1.2 Let Me Hear You Scream Ozzy Osbourne 2010 4 340 3.3 1
0.8 The Good Life Three Days Grace 2010 5 215 3.1 1.1
If the Red Hot Chili Peppers do have to pay a price for that panorama, it’s not showing up here. “Dani California” performed extremely well with an average grade level of 5.5. Three Days Grace even bucked their overall poor score with “Animal I Have Become.”
None of this, however, can undo what Ozzy hath wrought. “Let Me Hear You Scream” set a pretty low bar (1.2 to be exact). His other song, “I Don’t Wanna Stop,” didn’t do much better, either at 1.6.
Then again, it would be a crime not to mention Three Days Grace’s other record setting song: The Good Life. With an average grade level of 0.8, it was the lowest scoring song of the hundreds analyzed. Who said the good life was complicated?

Top Pop Songs Ranked by Grade Level

Average Grade Level Title Artist Year Weeks at #1 Word Count Syllables per Word Characters per Word
5 She Will Be Loved Maroon5 2004 4 351 1.2 3.7
4.8 E.T. Katy Perry Featuring Kanye West 2011 6 293 1.4 4.1
4.6 Hips Don’t Lie Shakira Featuring Wyclef Jean 2006 7 617 1.3 3.7
4.6 California Gurls Katy Perry Featuring Snoop Dogg 2010 7 411 1.4 4.2
4.5 We Are Young fun. Featuring Janelle Monae 2012 5 352 1.3 3.9
4.2 Shake It Off Mariah Carey 2005 5 352 1.2 3.6
4.2 Forever Chris Brown 2008 5 446 1.3 3.7
4.2 The Reason Hoobastank 2004 8 231 1.2 3.2
3.9 Irreplaceable Beyonce 2007 7 549 1.2 3.5
3.9 Leave (Get Out) JoJo 2004 5 330 1.1 3.4
3.7 We Belong Together Mariah Carey 2005 10 418 1.2 3.5
3.7 Boulevard Of Broken Dreams Green Day 2005 4 289 1.2 3.5
3.7 The Monster Eminem Featuring Rihanna 2014 5 710 1.2 3.7
3.6 Hollaback Girl Gwen Stefani 2005 6 459 1.2 3.4
3.5 Set Fire To The Rain Adele 2012 4 346 1.1 3.6
3.5 My Love Justin Timberlake Featuring T.I. 2006 4 784 1.2 3.4
3.5 Over And Over Nelly Featuring Tim McGraw 2004 8 546 1.2 3.3
3.4 Summer Love Justin Timberlake 2007 4 642 1.1 3.3
3.3 No One Alicia Keys 2008 5 314 1.2 3.2
3.3 Stronger (What Doesn’t Kill You) Kelly Clarkson 2012 4 429 1.2 3.7
3.3 Animals Maroon 5 2014 4 505 1.2 3.6
3.2 Big Girls Don’t Cry Fergie 2007 8 400 1.2 3.4
3.2 Leavin’ Jesse McCartney 2008 5 403 1.2 3.4
3.2 Bleeding Love Leona Lewis 2008 9 393 1.2 3.8
3.2 I Knew You Were Trouble. Taylor Swift 2013 7 588 1.2 3.5
3.2 One More Night Maroon 5 2012 8 492 1.2 3.4
3.1 Be Without You Mary J. Blige 2006 4 575 1.2 3.4
3.1 Low Flo Rida Featuring T-Pain 2008 6 731 1.1 3.7
3.1 All Of Me John Legend 2014 5 374 1.2 3.4
3 Roar Katy Perry 2013 5 445 1.1 3.2
3 Locked Out Of Heaven Bruno Mars 2013 4 323 1.1 3.5
2.9 Rolling In The Deep Adele 2011 5 574 1.2 3.6
2.9 Pieces Of Me Ashlee Simpson 2004 5 326 1.2 3.5
2.8 Teenage Dream Katy Perry 2010 4 421 1.2 3.5
2.8 We Found Love Rihanna Featuring Calvin Harris 2011 8 292 1.2 3.8
2.8 Check On It Beyonce Featuring Slim Thug 2006 6 617 1.1 3.2
2.8 Apologize Timbaland Featuring OneRepublic 2007 7 219 1.2 3.2
2.7 Firework Katy Perry 2011 4 391 1.2 3.6
2.7 Cupid’s Chokehold Gym Class Heroes Featuring Patrick Stump 2007 5 668 1.1 3.1
2.6 Telephone Lady Gaga Featuring Beyonce 2010 4 664 1.2 3.3
2.6 Payphone Maroon 5 Featuring Wiz Khalifa 2012 4 589 1.1 3.5
2.6 Blurred Lines Robin Thicke Featuring T.I. + Pharrell 2013 10 674 1.2 3.3
2.6 Dynamite Taio Cruz 2010 4 413 1.1 3.3
2.5 Toxic Britney Spears 2004 4 289 1.2 3.2
2.5 U + Ur Hand Pink 2007 4 478 1.2 3.5
2.5 Promiscuous Nelly Furtado Featuring Timbaland 2006 8 588 1.2 3.3
2.4 Can’t Hold Us Macklemore & Ryan Lewis Featuring Ray Dalton 2013 4 758 1.2 3.4
2.4 Happy Pharrell Williams 2014 5 544 1.2 3.5
2.3 Dark Horse Katy Perry Featuring Juicy J 2014 5 470 1.2 3.5
2.2 Say It Right Nelly Furtado 2007 4 247 1.1 3.2
2.2 I Gotta Feeling Black Eyed Peas 2009 7 646 1.2 3.2
2.1 Whatcha Say Jason DeRulo 2009 4 544 1.1 3.6
2.1 SexyBack Justin Timberlake 2006 5 644 1.2 3.3
2 Since U Been Gone Kelly Clarkson 2005 7 288 1.1 3.3
2 Boom Boom Pow The Black Eyed Peas 2009 7 526 1.1 3.6
1.9 Last Friday Night (T.G.I.F.) Katy Perry 2011 6 460 1.2 3.6
1.9 Grenade Bruno Mars 2011 5 447 1.1 3.3
1.9 Run It! Chris Brown 2005 4 706 1.1 3.3
1.8 Hey Ya! OutKast 2004 6 497 1.2 3.4
1.8 Gives You Hell The All-American Rejects 2009 4 463 1.1 3.4
1.7 Poker Face Lady GaGa 2009 5 573 1.1 2.9
1.5 Wide Awake Katy Perry 2012 4 354 1.2 3.4
1.5 TiK ToK Ke$ha 2010 7 499 1.1 3.4
1.4 So What Pink 2008 5 494 1.1 2.9
1.2 Moves Like Jagger Maroon 5 Featuring Christina Aguilera 2011 6 469 1.1 3.3
Now I finally arrive at Pop. Maroon 5 really put it best when they pointed out that “it’s not always rainbows and butterflies.” That line may have gotten them high points, but their odd tongue-centered request was enough to undo it all.
Katy Perry is in a similar position, with high and low scoring songs leaving her averaging out somewhere in a muddled middle. On the other hand, maybe we can just attribute her higher scoring songs to Snoop.
Pink on the other hand is a much simpler picture. True, her repeated use of words like “nah” puts her in good company (The Beatles, for example); but in terms of “smart” lyrics, she doesn’t do herself any favors.

Okay Children, What Have We Learned?

What have we learned here? Well, the concept of creating for the lowest common denominator is certainly going strong. So what if popular music doesn’t tend to lend itself to sophistication? Are most of these songs still great? Definitely.
Perhaps we can be a little less judgemental of elementary schoolers (you know who you are). It also wouldn’t hurt to be a little more judgemental of contemporary songwriters. More than anything, these findings are a reminder of just how fun dumb can be. In the end, there’s nothing wrong with that.

Friday, June 2, 2017

WannaCry Coding Mistakes Can Help Files Recovery Even After Infection

wannacry-ransomware-unlock-files-key
Last month WannaCry ransomware hit more than 300,000 PCs across the world within just 72 hours by using its self-spreading capabilities to infect vulnerable Windows PCs, particularly those using vulnerable versions of the OS, within the same network.

But that doesn't mean WannaCry was a high-quality piece of ransomware.

Security researchers have recently discovered some programming errors in the code of the WannaCrypt ransomware worm that might allow victims to restore their locked files without paying for any decryption key.

After deeply analysing the WannaCry code, security company at Kaspersky Lab found that the ransomware was full of mistakes that could allow some of its victims to restore their files with publicly available free recovery tools or even with simple commands.

Anton Ivanov, senior malware analyst at Kaspersky Lab, along with colleagues Fedor Sinitsyn and Orkhan Mamedov, detailed three critical errors made by WannaCry developers that could allow sysadmins to restore potentially lost files.

According to researchers, the issues reside in the way WannaCry ransomware deletes original files after encryption. In general, the malware first renames files to change their extension to ".WNCRYT," encrypt them and then delete the original files.

Recovering Read-only Files

unlock-wannacry-files
Since it is not at all possible for malicious software to directly encrypt or modify read-only files, WannaCry copies the files and creates their encrypted copies.

While the original files remain untouched but are given a 'hidden' attribute, getting the original data back simply requires victims to restore their normal attributes.

That wasn't the only mistake within the WannaCry's code, as in some cases, the malware fails to delete the files after encrypting them properly.

Recovering Files from the System Drive (i.e. C drive)


Researchers have said that files stored on the important folders, like Desktop or Documents folder, can not be recovered without the decryption key because WannaCry has been designed to overwrite original files with random data before removal.

However, researchers noticed that other files stored outside of important folders on the system drive could be restored from the temporary folder using a data recovery software.
“...the original file will be moved to %TEMP%\%d.WNCRYT (where %d denotes a numeric value). These files contain the original data and are not overwritten,” researchers said.


Recovering Files from the Non-System Drives

wannacry-recover-files
Researchers also found that for non-system drives, the WannaCry Ransomware creates a hidden '$RECYCLE' folder and moves original files into this directory after encryption. You can recover those files just by unhiding the '$RECYCLE' folder.

Also, due to "synchronization errors" in WannaCry's code, in many cases the original files remain in the same directory, making it possible for victims to restore insecurely deleted files using available data recovery software.

Programming Blunders: The New Hope for WannaCry Victims


These programming errors in the code of WannaCry offer hope to many victims.
"If you were infected with WannaCry ransomware there is a good possibility that you will be able to restore a lot of the files on the affected computer," Kaspersky Lab wrote in a blog post published Thursday. "The code quality is very low."
"To restore files, you can use the free utilities available for data recovery."
The recovery of files infected by WannaCry was first made possible by French researchers Adrien Guinet and Benjamin Delpy, who made a free WannaCry decryption tool that works on Windows XP, Windows 7, Windows Vista, Windows Server 2003 and Server 2008.

It's been almost a month since WannaCry epidemic hit computers worldwide, but the hackers behind the self-spread ransomware, which leverages leaked NSA's Windows SMB exploits EternalBlue and DoublePulsar, have not been identified yet.

While police and cyber security firms continue to search for answers surrounding the origins of the WannaCry campaign, Dark web intelligence firm Flashpoint recently indicated the perpetrators might be Chinese, based on its linguistic analysis.

Friday, May 5, 2017

Beware! Don't Fall for FireFox "HoeflerText Font Wasn't Found" Banking Malware Scam


firefox-chrome-HoeflerText-font-was-not-found
The malicious scam campaign, "The 'HoeflerText' font wasn't found," is back, which was previously targeting Google Chrome users to trick them into installing Spora ransomware on their computers.

This time the campaign has been re-designed to target Mozilla Firefox users with a banking trojan, called Zeus Panda.

Interestingly, the attackers behind this new campaign are so stupid that they forgot to change the name of the font, i.e. HoeflerText, due to which it was easily caught by Kafeine, a security researcher at Proofpoint.

As I previously warned — Next time when you accidentally land up on a suspicious website with jumbled content prompting to update the Firefox or Chrome font pack by downloading a missing text font to read the article… Just don't download it. It's obviously a trap.

Just like the previous one, the latest Firefox 'HoeflerText font wasn't found scam is also very convincing and easy to fall for.

The attack initiates with an alert message, which states that "The 'HoeflerText' font was not found," asking Firefox users to update their "Mozilla Font Pack."

Once clicked, it downloads a ZIP file (Mozilla_Font_v7.87.zip) on the victim’s system, containing a JavaScript file. Meanwhile, the screen will display a set of instructions, asking victims to run the JS file in order to install the missing "Mozilla Font Pack."

If the victims turned out to be so stupid that they run the JS file themselves, as instructed, the program will download the malware payload (.exe) from a remote server and then runs it automatically, injecting the Zeus Panda banking Trojan into the infected systems.

Panda is an online banking trojan that was initially found to be targeting banks in Europe and North America early last year and then started spreading itself to Brazil through at least three different exploit kits, including Angler, Nuclear, and Neutrino.

Once it infects a system, the Zeus Panda banking trojan contacts its command and control (C&C) server to send the infected device information, including a list of installed antivirus and firewall products.

Panda focuses on stealing banking credentials from users, as well as those of bitcoin exchanges, payment card services and online payments providers, prepaid cards, airline loyalty programs and online betting accounts, to name a few.

To protect yourself from such scams, always exercise caution when downloading anything from the Internet onto your computers. Moreover, keep your antivirus up-to-date and do not ever fall for scam asking you to update your Mozilla or Chrome font pack, as it already comes with everything you need. 

Wang Wei

Saturday, April 29, 2017

Insecure Apps that Open Ports Leave Millions of Smartphones at Risk of Hacking



Thursday, April 20, 2017

The Lack of Food is Not the Cause of the Hunger Problem


The problem has never been the lack of food - but the access to food.
Neel Ghose | TEDxGateway 

Neel is the founder of the Robin Hood Army. Being a finance student and having worked in a New York based hedge fund (D.E. Shaw), Neel looked set for a career in finance – only to realise that he found excel sheets and financial analysis exceptionally boring. This led him to the world of food and startups, and he joined a small passionate team that believed in building a world-class product. Neel is a bit of a nomad and has lived in 8 cities across 5 countries in the last couple of years, setting up operations of the startup Zomato, worldwide.
In his free time, along with his friend Anand, Neel has set up the Robin Hood Army - a volunteer based organisation which collects excess food from restaurants and distributes it to the less fortunate. In a little over two years, the RHA has served over 1.5 million people through over 8000 Robins across 12 countries. He likes to believe that the RHA is just '1% Done'.

This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at http://ted.com/tedx

Monday, April 10, 2017

Smartphones Using Broadcom Wi-Fi Chip Can Be Hacked OTA


Broadcom-wifi-chipset-hack
Millions of smartphones and smart gadgets, including Apple iOS and many Android handsets from various manufacturers, equipped with Broadcom Wifi chips are vulnerable to over-the-air hijacking without any user interaction.

Just yesterday, Apple rushed out an emergency iOS 10.3.1 patch update to address a serious bug that could allow an attacker within same Wifi network to remotely execute malicious code on the Broadcom WiFi SoC (Software-on-Chip) used in iPhones, iPads, and iPods.

The vulnerability was described as the stack buffer overflow issue and was discovered by Google's Project Zero staffer Gal Beniamini, who today detailed his research on a lengthy blog post, saying the flaw affects not only Apple but all those devices using Broadcom's Wi-Fi stack.

Beniamini says this stack buffer overflow issue in the Broadcom firmware code could lead to remote code execution vulnerability, allowing an attacker in the smartphone's WiFi range to send and execute code on the device.

Attackers with high skills can also deploy malicious code to take full control over the victim's device and install malicious apps, like banking Trojans, ransomware, and adware, without the victim's knowledge.

In his next blog post that's already on its way, Beniamini will explain how attackers can use their assumed control of the Wi-Fi SoC in order to further escalate their privileges into the application processor, taking over the host’s operating system.

Over-the-Air Broadcom Wi-Fi SoC Hack

hack-android-phone
According to the researcher, the firmware running on Broadcom WiFi SoC can be tricked into overrunning its stack buffers, which allowed him to send carefully crafted WiFi frames, with abnormal values, to the Wi-Fi controller in order to overflow the firmware's stack.

Beniamini then combined this value with the frequent timer firings of the chipset to gradually overwrite specific chunks of device's memory (RAM) until his malicious code is executed.

So, to exploit the flaw, an attacker needs to be within the WiFi range of the affected device to silently take over it.
"While the firmware implementation on the Wi-Fi SoC is incredibly complex, it still lags behind in terms of security," Beniamini explains. "Specifically, it lacks all basic exploit mitigations – including stack cookies, safe unlinking and access permission protection."
The researcher also detailed a proof-of-concept Wi-Fi remote code execution exploit in the blog post and successfully performed it on a then-fully updated (now fixed) Nexus 6P, running Android 7.1.1 version NUF26K – the latest available Nexus device at the time of testing in February.

The flaw is one of the several vulnerabilities discovered by Beniamini in the firmware version 6.37.34.40 of Broadcom Wi-Fi chips.

Security Patch for Nexus & iOS Released; Others Have to Wait!


Google Project Zero team reported the issue to Broadcom in December. Since the flaw is in Broadcom's code, smartphone makers had to wait for a patch from the chip vendor before testing the patch and pushing it out to their own user base.

Both Apple and Google addressed the vulnerability with security updates released on Monday, with Google delivering updates via its Android April 2017 Security Bulletin and Apple releasing the iOS 10.3.1 update.

The flaw still affects most Samsung flagship devices, including Galaxy S7 (G930F, G930V), Galaxy S7 Edge (G935F, G9350), Galaxy S6 Edge (G925V), Galaxy S5 (G900F), and Galaxy Note 4 (N910F), the researcher says.
by

Longhorn: Tools used by cyber-espionage group linked to Vault 7

First evidence linking Vault 7 tools to known cyberattacks.

Spying tools and operational protocols detailed in the recent Vault 7 leak have been used in cyberattacks against at least 40 targets in 16 different countries by a group Symantec calls Longhorn. Symantec has been protecting its customers from Longhorn’s tools for the past three years and has continued to track the group in order to learn more about its tools, tactics, and procedures.
The tools used by Longhorn closely follow development timelines and technical specifications laid out in documents disclosed by WikiLeaks. The Longhorn group shares some of the same cryptographic protocols specified in the Vault 7 documents, in addition to following leaked guidelines on tactics to avoid detection. Given the close similarities between the tools and techniques, there can be little doubt that Longhorn's activities and the Vault 7 documents are the work of the same group.

Who is Longhorn?

Longhorn has been active since at least 2011. It has used a range of back door Trojans in addition to zero-day vulnerabilities to compromise its targets. Longhorn has infiltrated governments and internationally operating organizations, in addition to targets in the financial, telecoms, energy, aerospace, information technology, education, and natural resources sectors. All of the organizations targeted would be of interest to a nation-state attacker.
Longhorn has infected 40 targets in at least 16 countries across the Middle East, Europe, Asia, and Africa. On one occasion a computer in the United States was compromised but, following infection, an uninstaller was launched within hours, which may indicate this victim was infected unintentionally.
#Vault7 linked #Longhorn group infiltrated governments, international orgs, other targets Click to Tweet

The link to Vault 7

A number of documents disclosed by WikiLeaks outline specifications and requirements for malware tools. One document is a development timeline for a piece of malware called Fluxwire, containing a changelog of dates for when new features were incorporated. These dates align closely with the development of one Longhorn tool (Trojan.Corentry) tracked by Symantec. New features in Corentry consistently appeared in samples obtained by Symantec either on the same date listed in the Vault 7 document or several days later, leaving little doubt that Corentry is the malware described in the leaked document.
Early versions of Corentry seen by Symantec contained a reference to the file path for the Fluxwire program database (PDB) file. The Vault 7 document lists removal of the full path for the PDB as one of the changes implemented in Version 3.5.0.
Up until 2014, versions of Corentry were compiled using GCC. According to the Vault 7 document, Fluxwire switched to a MSVC compiler for version 3.3.0 on February 25, 2015. This was reflected in samples of Corentry, where a version compiled on February 25, 2015 had used MSVC as a compiler.
Corentry sample (MD5 hash) Date/time of sample compilation Embedded Corentry version number Corentry compiler Vault 7 changelog number Vault 7 changelog date
N/A N/A N/A N/A 2.1.0 - 2.4.1 Jan 12, 2011 - Feb 28, 2013
e20d5255d8ab1ff5f157847d2f3ffb25 23/08/2013 10:20 3.0.0 GCC 3.0.0 Aug 23, 2013
5df76f1ad59e019e52862585d27f1de2 21/02/2014 11:07 3.1.0 GCC 3.1.0 Feb 20, 2014
318d8b61d642274dd0513c293e535b38 15/05/2014 09:01 3.1.1 GCC 3.1.1 May 14, 2014
N/A N/A N/A N/A 3.2.0 Jul 15, 2014
511a473e26e7f10947561ded8f73ffd0 03/09/2014 00:12 3.2.1 GCC 3.2.1 Aug 18, 2014
c06d422656ca69827f63802667723932 25/02/2015 16:50 N/A MSVC 3.3.0 Feb 25, 2015
N/A N/A N/A N/A 3.3.1 -> 3.5.0 May 17, 2015 -> Nov 13, 2015
Table. Corentry version numbers and compilation dates compared to Fluxwire version numbers and changelog dates disclosed in Vault 7
A second Vault 7 document details Fire and Forget, a specification for user-mode injection of a payload by a tool called Archangel. The specification of the payload and the interface used to load it was closely matched in another Longhorn tool called Backdoor.Plexor.
A third document outlines cryptographic protocols that malware tools should follow. These include the use of inner cryptography within SSL to prevent man-in-the-middle (MITM) attacks, key exchange once per connection, and use of AES with a 32-bit key. These requirements align with the cryptographic practices observed by Symantec in all of the Longhorn tools.
Other Vault 7 documents outline tradecraft practices to be used, such as use of the Real-time Transport Protocol (RTP) as a means of command and control (C&C) communications, employing wipe-on-use as standard practice, in-memory string de-obfuscation, using a unique deployment-time key for string obfuscation, and the use of secure erase protocols involving renaming and overwriting. Symantec has observed Longhorn tools following all of these practices. While other malware families are known to use some of these practices, the fact that so many of them are followed by Longhorn makes it noteworthy.

Global reach: Longhorn’s operations

While active since at least 2011, with some evidence of activity dating back as far as 2007, Longhorn first came to Symantec’s attention in 2014 with the use of a zero-day exploit (CVE-2014-4148) embedded in a Word document to infect a target with Plexor.
The malware had all the hallmarks of a sophisticated cyberespionage group. Aside from access to zero-day exploits, the group had preconfigured Plexor with a proxy address specific to the organization, indicating that they had prior knowledge of the target environment.
To date, Symantec has found evidence of Longhorn activities against 40 targets spread across 16 different countries. Symantec has seen Longhorn use four different malware tools against its targets: Corentry, Plexor, Backdoor.Trojan.LH1, and Backdoor.Trojan.LH2.
Before deploying malware to a target, the Longhorn group will preconfigure it with what appears to be target-specific code words and distinct C&C domains and IP addresses for communications back to the attackers. Longhorn tools have embedded capitalized code words, internally referenced as “groupid” and “siteid”, which may be used to identify campaigns and victims. Over 40 of these identifiers have been observed, and typically follow the theme of movies, characters, food, or music. One example was a nod to the band The Police, with the code words REDLIGHT and ROXANNE used.
Longhorn’s malware has an extensive list of commands for remote control of the infected computer. Most of the malware can also be customized with additional plugins and modules, some of which have been observed by Symantec.
Longhorn’s malware appears to be specifically built for espionage-type operations, with detailed system fingerprinting, discovery, and exfiltration capabilities. The malware uses a high degree of operational security, communicating externally at only select times, with upload limits on exfiltrated data, and randomization of communication intervals—all attempts to stay under the radar during intrusions.
For C&C servers, Longhorn typically configures a specific domain and IP address combination per target. The domains appear to be registered by the attackers; however they use privacy services to hide their real identity. The IP addresses are typically owned by legitimate companies offering virtual private server (VPS) or webhosting services. The malware communicates with C&C servers over HTTPS using a custom underlying cryptographic protocol to protect communications from identification.
Prior to the Vault 7 leak, Symantec’s assessment of Longhorn was that it was a well-resourced organization which was involved in intelligence gathering operations. This assessment was based on its global range of targets and access to a range of comprehensively developed malware and zero-day exploits. The group appeared to work a standard Monday to Friday working week, based on timestamps and domain name registration dates, behavior which is consistent with state-sponsored groups.
Symantec’s analysis uncovered a number of indicators that Longhorn was from an English-speaking, North American country. The acronym MTWRFSU (Monday Tuesday Wednesday ThuRsday Friday Saturday SUnday) was used to configure which day of the week malware would communicate with the attackers. This acronym is common in academic calendars in North America. Some of the code words found in the malware, such as SCOOBYSNACK, would be most familiar in North America. In addition to this, the compilation times of tools with reliable timestamps indicate a time zone in the Americas.

Distinctive fingerprints

Longhorn has used advanced malware tools and zero-day vulnerabilities to infiltrate a string of targets worldwide. Taken in combination, the tools, techniques, and procedures employed by Longhorn are distinctive and unique to this group, leaving little doubt about its link to Vault 7.
Throughout its investigation of Longhorn, Symantec’s priority has been protection of its customers. Through identifying different strains of Longhorn malware, connecting them to a single actor, and learning more about the group’s tactics and procedures, Symantec has been able to better defend customer organizations against this and similar threats. In publishing this new information, Symantec’s goal remains unchanged: to reassure customers that it is aware of this threat and actively working to protect them from it.

Protection

Symantec and Norton products protect against Longhorn malware with the following detections:

 By: Symantec Security Response Symantec Employee