Web Scraping of English Premier League Match Data in R

Big Data and Web Scraping

According to some experts, the amount of data in the world is expected to grow by over 60% per year between now and 2025. It is also predicted that 80% of the world’s data will be unstructured by 2025. Structured data is data that is organised in a predefined structure such as in tables. Unstructured data is the rest. Thus, for instance, a MS Excel spreadsheet would typically contain structured data, whereas a Twitter feed would typically contain unstructured data.

In this article we will introduce the reader to web scraping, which refers to extracting data from the World Wide Web and (where necessary) cleaning the data to get it in a usable (e.g. structured) format. In light of the above predictions, it is clear that web scraping will be an increasingly useful skill going forward.

Before going further, an ethical disclaimer is needed. Web scraping could be a tool in the toolkit of an unscrupulous hacker. However, what is being advocated here is not stealing secure, proprietary data but rather harvesting publicly available data. The data we will be scraping is already accessible to anyone with an Internet connection and a web browser; we are just accessing it in a more efficient and systematic way.

Harvesting HTML-Rendered Content in R

In order to proceed you will need to download R and ideally RStudio as your programming user interface.

We will see that you can get up and running as a web scraper in R very quickly, without a lot of expertise either in how the Internet works or in R programming. This is because a lot of the functionality is available in ready-made R packages. I will attempt to very briefly explain how web scraping works, and it will necessarily be in layman’s terms because I am no expert in it myself.

The basic structure of most websites is written in HTML code. When you load a webpage in your browser, your browser compiles the HTML code and displays the webpage to you. Traditionally, any data that existed on the webpage (e.g., tables, numbers, text) was already present in the HTML code, but wrapped in various ‘tags’ that would determine how the content is formatted. For instance, if the text was to be displayed as a clickable link, it would be wrapped in <a> </a>.

The R packages rvest and xml2 contain the functions we need to very easily extract the HTML code and HTML-rendered content from any publicly accessible webpage. We first need to install the rvest package (which will automatically result in xml2 also being installed.) We will also need the xlsx package to write our results to an Excel spreadsheet.

install.packages("rvest")
install.packages("xlsx")

Then we can call the function read_html from xml2 package and send it any webpage URL, and it will save the HTML code in an R object.

A Basic Example: Google South Africa Homepage

As a first example, let us scrape the Google South Africa homepage.

myurl <- "https://www.google.co.za"
mygoogle <- xml2::read_html(myurl)

The object mygoogle now stores the HTML code for the Google homepage. A simple way to view the content we have scraped is using the function rvest::html_text, which eliminates the HTML tags and preserves the content in between them as raw text in a character. If we do this with the Google homepage and display the result, it is not very impressive: we basically get a lot of what looks like gibberish.

googlechar <- rvest::html_text(mygoogle)
googlechar

## [1] "Google(function(){window.google={kEI:'wdeeXoHtEI6bkwWcx6DoBA',kEXPI:'31',kBL:'2amb'};google.sn='webhp';google.kHL='en-ZA';})();(function(){google.lc=[];google.li=0;google.getEI=function(a){for(var c;a&&(!a.getAttribute||!(c=a.getAttribute(\"eid\")));)a=a.parentNode;return c||google.kEI};google.getLEI=function(a){for(var c=null;a&&(!a.getAttribute||!(c=a.getAttribute(\"leid\")));)a=a.parentNode;return c};google.ml=function(){return null};google.time=function(){return Date.now()};google.log=function(a,c,b,d,g){if(b=google.logUrl(a,c,b,d,g)){a=new Image;var e=google.lc,f=google.li;e[f]=a;a.onerror=a.onload=a.onabort=function(){delete e[f]};google.vel&&google.vel.lu&&google.vel.lu(b);a.src=b;google.li=f+1}};google.logUrl=function(a,c,b,d,g){var e=\"\",f=google.ls||\"\";b||-1!=c.search(\"&ei=\")||(e=\"&ei=\"+google.getEI(d),-1==c.search(\"&lei=\")&&(d=google.getLEI(d))&&(e+=\"&lei=\"+d));d=\"\";!b&&google.cshid&&-1==c.search(\"&cshid=\")&&\"slh\"!=a&&(d=\"&cshid=\"+google.cshid);b=b||\"/\"+(g||\"gen_204\")+\"?atyp=i&ct=\"+a+\"&cad=\"+c+e+f+\"&zx=\"+google.time()+d;/^http:/i.test(b)&&\"https:\"==window.location.protocol&&(google.ml(Error(\"a\"),!1,{src:b,glmm:1}),b=\"\");return b};}).call(this);(function(){google.y={};google.x=function(a,b){if(a)var c=a.id;else{do c=Math.random();while(google.y[c])}google.y[c]=[a,b];return!1};google.lm=[];google.plm=function(a){google.lm.push.apply(google.lm,a)};google.lq=[];google.load=function(a,b,c){google.lq.push([[a],b,c])};google.loadAll=function(a,b){google.lq.push([a,b])};}).call(this);google.f={};(function(){\ndocument.documentElement.addEventListener(\"submit\",function(b){var a;if(a=b.target){var c=a.getAttribute(\"data-submitfalse\");a=\"1\"==c||\"q\"==c&&!a.elements.q.value?!0:!1}else a=!1;a&&(b.preventDefault(),b.stopPropagation())},!0);document.documentElement.addEventListener(\"click\",function(b){var a;a:{for(a=b.target;a&&a!=document.documentElement;a=a.parentElement)if(\"A\"==a.tagName){a=\"1\"==a.getAttribute(\"data-nohref\");break a}a=!1}a&&b.preventDefault()},!0);}).call(this);\nvar a=window.location,b=a.href.indexOf(\"#\");if(0<=b){var c=a.href.substring(b+1);/(^|&)q=/.test(c)&&-1==c.indexOf(\"#\")&&a.replace(\"/search?\"+c.replace(/(^|&)fp=[^&]*/g,\"\")+\"&cad=h\")};#gbar,#guser{font-size:13px;padding-top:1px !important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:right}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;vertical-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{color:#900 !important}\nbody,td,a,p,.h{font-family:arial,sans-serif}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}.ts td{padding:0}.ts{border-collapse:collapse}em{font-weight:bold;font-style:normal}.lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17px arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;margin-left:4px}input{font-family:inherit}body{background:#fff;color:#000}a{color:#11c;text-decoration:none}a:hover,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}.sblc{padding-top:5px}.sblc a{display:block;margin:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;border-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}.ftl,#fll a{display:inline-block;margin:0 12px}.lsb{background:url(/images/nav_logo229.png) 0 -261px repeat-x;border:none;color:#000;cursor:pointer;height:30px;margin:0;outline:0;font:15px arial,sans-serif;vertical-align:top}.lsb:active{background:#ccc}.lst:focus{outline:none}(function(){var src='/images/nav_logo229.png';var iesg=false;document.body.onload = function(){window.n && window.n();if (document.images){new Image().src=src;}\nif (!iesg){document.f&&document.f.q.focus();document.gbqf&&document.gbqf.q.focus();}\n}\n})(); Search Images Maps Play YouTube News Gmail Drive More »Web History | Settings | Sign in  (function(){var id='tsuid1';document.getElementById(id).onclick = function(){if (this.form.q.value){this.checked = 1;if (this.form.iflsig)this.form.iflsig.disabled = false;}\nelse top.location='/doodles/';};})();Advanced search(function(){var a,b=\"1\";if(document&&document.getElementById)if(\"undefined\"!=typeof XMLHttpRequest)b=\"2\";else if(\"undefined\"!=typeof ActiveXObject){var c,d,e=[\"MSXML2.XMLHTTP.6.0\",\"MSXML2.XMLHTTP.3.0\",\"MSXML2.XMLHTTP\",\"Microsoft.XMLHTTP\"];for(c=0;d=e[c++];)try{new ActiveXObject(d),b=\"2\"}catch(h){}}a=b;if(\"2\"==a&&-1==location.search.indexOf(\"&gbv=2\")){var f=google.gbvu,g=document.getElementById(\"gbv\");g&&(g.value=a);f&&window.setTimeout(function(){location.href=f},0)};}).call(this);.szppmdbYutt__middle-slot-promo{font-size:small;margin-bottom:32px}.szppmdbYutt__middle-slot-promo a.ZIeIlb{display:inline-block;text-decoration:none}.szppmdbYutt__middle-slot-promo img{border:none;margin-right:5px;vertical-align:middle}Stay Home. Save Lives#gws-output-pages-elements-homepage_additional_languages__als{font-size:small;margin-bottom:24px}#SIvCob{display:inline-block;line-height:28px;}#SIvCob a{padding:0 3px;}.H6sW5{display:inline-block;margin:0 2px;white-space:nowrap}.z4hgWe{display:inline-block;margin:0 2px}Google offered in:  Afrikaans    Sesotho    isiZulu    IsiXhosa    Setswana    Northern Sotho  Advertising<U+00A0>ProgramsBusiness SolutionsAbout GoogleGoogle.com© 2020 - Privacy - Terms(function(){window.google.cdo={height:0,width:0};(function(){var a=window.innerWidth,b=window.innerHeight;if(!a||!b){var c=window.document,d=\"CSS1Compat\"==c.compatMode?c.documentElement:c.body;a=d.clientWidth;b=d.clientHeight}a&&b&&(a!=google.cdo.width||b!=google.cdo.height)&&google.log(\"\",\"\",\"/client_204?&atyp=i&biw=\"+a+\"&bih=\"+b+\"&ei=\"+google.kEI);}).call(this);})();(function(){google.kEXPI='0,202123,3,1151620,5663,730,224,5104,207,3204,10,1051,175,364,925,510,4,60,239,337,241,309,74,246,5,860,99,51,9,335,108,88,334,42,88,108,45,337,34,138,5,89,278,98,98,419133,706743,1197733,416,329118,1294,12383,4855,32691,15248,864,28687,369,8819,8384,4859,996,365,9290,3023,4745,3118,7915,1808,4020,978,7931,5192,103,2056,920,873,1217,2382,593,2784,3646,11306,2902,319,4518,2777,520,399,2277,8,2796,1593,1279,390,1822,530,149,1103,841,518,1137,1,277,57,48,158,4100,312,1136,3,2063,606,1839,184,1777,143,377,1947,245,502,1482,93,328,1284,17,446,2480,2246,474,1339,29,719,1039,3229,2843,7,438,379,4782,8547,2662,642,1407,1042,2459,1226,1462,3935,1274,108,1712,1697,906,2,940,533,313,1769,2397,1387,3583,449,226,657,338,830,840,480,606,1349,3,12,334,201,29,157,813,865,378,1634,1908,438,266,149,189,354,2959,502,1,1539,47,268,131,28,130,1,70,795,1228,2598,1345,46,1093,82,569,4,1337,191,17,619,1,22,283,69,4,279,381,629,152,65,243,566,188,22,271,128,95,367,284,385,2,18,5,2,12,22,64,129,941,413,272,1817,278,211,120,174,142,525,118,94,15,117,573,127,4,634,15,6,169,237,19,126,336,702,724,88,165,43,834,678,1131,788,5824815,3277,32,1802585,6996022,549,333,444,1,2,80,1,900,896,1,8,1,2,2551,1,748,141,59,736,563,1,4265,1,1,1,1,137,1,1193,1259,142,3,5,91,63,4,109,1,46,7,4,2,10,6,1,1,36,1,1,1,3,20742588,3220019';})();(function(){var u='/xjs/_/js/k\\x3dxjs.hp.en.8jTnuCMMiFI.O/m\\x3dsb_he,d/am\\x3dAAMCbAQ/d\\x3d1/rs\\x3dACT90oFSe3F4u6cscTHTUw58MPqxYvNkww';\nsetTimeout(function(){var b=document;var a=\"SCRIPT\";\"application/xhtml+xml\"===b.contentType&&(a=a.toLowerCase());a=b.createElement(a);a.src=u;google.timers&&google.timers.load&&google.tick&&google.tick(\"load\",\"xjsls\");document.body.appendChild(a)},0);})();(function(){window.google.xjsu='/xjs/_/js/k\\x3dxjs.hp.en.8jTnuCMMiFI.O/m\\x3dsb_he,d/am\\x3dAAMCbAQ/d\\x3d1/rs\\x3dACT90oFSe3F4u6cscTHTUw58MPqxYvNkww';})();function _DumpException(e){throw e;}\nfunction _F_installCss(c){}\n(function(){google.jl={em:[],emw:false,lls:'default',pdt:0,snet:true,uwp:true};})();(function(){var pmc='{\\x22d\\x22:{},\\x22sb_he\\x22:{\\x22agen\\x22:true,\\x22cgen\\x22:true,\\x22client\\x22:\\x22heirloom-hp\\x22,\\x22dh\\x22:true,\\x22dhqt\\x22:true,\\x22ds\\x22:\\x22\\x22,\\x22ffql\\x22:\\x22en\\x22,\\x22fl\\x22:true,\\x22host\\x22:\\x22google.co.za\\x22,\\x22isbh\\x22:28,\\x22jsonp\\x22:true,\\x22msgs\\x22:{\\x22cibl\\x22:\\x22Clear Search\\x22,\\x22dym\\x22:\\x22Did you mean:\\x22,\\x22lcky\\x22:\\x22I\\\\u0026#39;m Feeling Lucky\\x22,\\x22lml\\x22:\\x22Learn more\\x22,\\x22oskt\\x22:\\x22Input tools\\x22,\\x22psrc\\x22:\\x22This search was removed from your \\\\u003Ca href\\x3d\\\\\\x22/history\\\\\\x22\\\\u003EWeb History\\\\u003C/a\\\\u003E\\x22,\\x22psrl\\x22:\\x22Remove\\x22,\\x22sbit\\x22:\\x22Search by image\\x22,\\x22srch\\x22:\\x22Google Search\\x22},\\x22ovr\\x22:{},\\x22pq\\x22:\\x22\\x22,\\x22refpd\\x22:true,\\x22rfs\\x22:[],\\x22sbpl\\x22:16,\\x22sbpr\\x22:16,\\x22scd\\x22:10,\\x22stok\\x22:\\x22NyTRBYu_yyJ-6iCGKbuJZboi-S4\\x22,\\x22uhde\\x22:false}}';google.pmc=JSON.parse(pmc);})();"

This is a reality of web scraping: we are always going to have a lot of useless data mixed in with the useful data in our output, so we need ways of separating the wheat from the chaff. In R, if we have converted the HTML document to raw text, we can use string manipulation functions such as gregexpr and substr to locate and extract the useful content. Alternatively, we can use the function html_nodes in the rvest package to locate content stored within a particular HTML tag. As a guide to doing so, it is useful to open the page and its source code in our web browser. In Google Chrome, we can visit the Google homepage and then press Ctrl+Shift+i to open Developer tools. Then we can scroll through the HTML code and see the HTML tags surrounding various content.

For example, suppose we want to know the languages in which Google is available in South Africa. Using the raw text approach, we notice that the words ‘Google offered in’ appear before these languages. We can first remove all characters other than alphanumeric, punctuation, and spaces (otherwise some weird special characters will result in an error), then search for the location of the expression ‘Google offered in’ and take a substring beginning from that location until 100 characters later.

googlechar2 <- gsub("[^[:alnum:][:punct:][:space:]]", "", googlechar)
textlocation <- gregexpr("Google offered in", googlechar2)[[1]]
substr(googlechar2, start = textlocation, stop = textlocation + 120)

## [1] "Google offered in:  Afrikaans    Sesotho    isiZulu    IsiXhosa    Setswana    Northern Sotho  AdvertisingProgramsBusines"

Voilà, the list of languages. Now, if instead we wanted to extract the title of the webpage, the html_text approach would be less effective. The word ‘title’ appears nowhere in the resulting string, even after putting it into lowercase. (Note that gregexpr returns -1 when it does not find the expression in the string).

gregexpr("title", tolower(googlechar2))[[1]]

## [1] -1
## attr(,"match.length")
## [1] -1

If we know a little bit of HTML, or otherwise if we inspect the HTML code in Google Chrome Developer tools, we will observe that the webpage title appears within a <title> tag. The tags are eliminated when we call html_text. However, we can instead call html_nodes, which requests all instances of a specific tag or node. Then we run html_text on the result, to get the text within the <title> tag.

titletag <- rvest::html_nodes(mygoogle, "title")
titletag

## {xml_nodeset (1)}
## [1] <title>Google</title>\n

rvest::html_text(titletag)

## [1] "Google"

And voilà! The title of the page is ‘Google’.

Scraping Barclays Premier League Match Data for 2019-2020

Okay, scraping the Google homepage is not that interesting. However, for fans of English football, scraping all the match data for the 2019-2020 Premier League season may provide temporary relief of withdrawal symptoms. First, we need a website that stores the match data in static HTML format. Fox Sports Australia does the trick. You can see on this page the result of every match during the season, and can filter by team and/or round (matchday no.). If you hover over a particular match result, you can see a link to a Match Centre page with more detailed results, such as Liverpool’s 4-1 defeat of Norwich City on 10 August 2019. There, on the Stats tab, we can see various statistics for both teams such as Possession, Shots, Passing, etc. At the bottom we can also see various statistics for each individual player in a table, one for each team. Our goal here is to create a spreadsheet storing the Match Centre statistics for all of the matches played this season.

totmatchdays <- 29
ro <- c(paste0("0", 1:9), 10:totmatchdays)
ma <- c(paste0("0", 1:9), 10)
nmatch <- totmatchdays * 10
urls <- rep(NA, nmatch)
fullhtml <- vector("list", nmatch)

i <- 0
for (r in ro) {
   for (m in ma) {
      i <- i + 1
      print(i)
      if (r == "18" && m == "06") {
       urls[i] <- "https://www.foxsports.com.au/football/premier-league/match-centre/EPL2019-202511/stats"
      } else if (r == "26" && m == "05") {
       urls[i] <- "https://www.foxsports.com.au/football/premier-league/match-centre/EPL2019-202611/stats"
      } else {
       urls[i] <- paste0("https://www.foxsports.com.au/football/premier-league/match-centre/EPL2019-20", 
                            r, m, "/stats")      
      }

    if (!(r == "28" && (m == "01" || m == "03"))) {
     loadedcorrectly <- FALSE
     while (loadedcorrectly == FALSE) {
        fullhtml[[i]] <- xml2::read_html(urls[i])
        titlenodes <- rvest::html_nodes(fullhtml[[i]], "title")
        figcaptionnodes <- rvest::html_nodes(fullhtml[[i]], "figcaption")
        if (length(titlenodes) == 2 && length(figcaptionnodes) == 25) loadedcorrectly <- TRUE
     }
    } 
   }
}

  i <- 1 # For first match only
  mytitle <- rvest::html_text(rvest::html_nodes(fullhtml[[i]], "title"))
  hometeam <- substr(mytitle[1], start = 1, stop = gregexpr("vs", mytitle[1])[[1]] - 2)
  awayteam <- substr(mytitle[1], start = gregexpr("vs", mytitle[1])[[1]] + 3, 
                     stop = gregexpr("Live Statistics", mytitle[1])[[1]] - 2)

  mydivs <- rvest::html_nodes(fullhtml[[i]], "div")
  mydivclass <- rvest::html_attr(mydivs, name = "class")
  mydivtext <- rvest::html_text(mydivs)
  matchscorelocation <- which(gregexpr("MatchScore", mydivclass) != -1)
  scores <- mydivtext[matchscorelocation]
  scores

## [1] "4" "1"

  possessionlocation <- which(gregexpr("TeamLabel", mydivclass) != -1)
  possession <- mydivtext[possessionlocation]
  possession

## [1] "55%" "45%"

  territorylocation <- which(gregexpr("StatsComparisonBar-ar67kb", mydivclass) != -1)
  territory <- mydivtext[territorylocation]
  territory

## [1] "60%" "40%"

  myfigcaptions <- rvest::html_nodes(fullhtml[[i]], "figcaption")
  otherstats <- rvest::html_text(myfigcaptions)[-1]
  otherstats

##  [1] "7"   "3"   "5"   "5"   "3"   "2"   "392" "284" "11"  "2"   "1"   "10"  "8"   "30" 
## [15] "2"   "4"   "0"   "5"   "9"   "9"   "0"   "2"   "0"   "0"

matchdata <- matrix(nrow = 0, ncol = 19)
for (i in setdiff(1:nmatch, c(271, 273))) {
  
  # Get name of home team and away team
  mytitle <- rvest::html_text(rvest::html_nodes(fullhtml[[i]], "title"))
  hometeam <- substr(mytitle[1], start = 1, stop = gregexpr("vs", mytitle[1])[[1]] - 2)
  awayteam <- substr(mytitle[1], start = gregexpr("vs", mytitle[1])[[1]] + 3, 
                     stop = gregexpr("Live Statistics", mytitle[1])[[1]] - 2)

  # Error checking  
  if (hometeam == "" || awayteam == "") {
    print(i)
    stop(paste0(i, "th Team Name missing! Run `fullhtml[[i]] <- xml2::read_html(urls[i])` again for current i"))
  }
  
  # Get scores of home team and away team
  mydivs <- rvest::html_nodes(fullhtml[[i]], "div")
  mydivclass <- rvest::html_attr(mydivs, name = "class")
  mydivtext <- rvest::html_text(mydivs)
  matchscorelocation <- which(gregexpr("MatchScore", mydivclass) != -1)
  scores <- mydivtext[matchscorelocation]

  # Get possession and territory values of home team and away team  
  possessionlocation <- which(gregexpr("TeamLabel", mydivclass) != -1)
  possession <- mydivtext[possessionlocation]
  territorylocation <- which(gregexpr("StatsComparisonBar-ar67kb", mydivclass) != -1)
  territory <- mydivtext[territorylocation]
  
  # Error checking
  if (length(possession) == 0 || length(territory) == 0) stop("possession or territory empty! Run `fullhtml[[i]] <- xml2::read_html(urls[i])` again for current i")
  
  # Get other 12 team statistics for home and away team
  myfigcaptions <- rvest::html_nodes(fullhtml[[i]], "figcaption")
  otherstats <- rvest::html_text(myfigcaptions)[-1]
  
  # Error checking
  if (length(otherstats) == 0) stop("otherstats empty! Run `fullhtml[[i]] <- xml2::read_html(urls[i])` again for current i")
  
  # Get matchday (round) number from URL
  matchday <- substr(urls[i], start = 77, stop = 78)

  # Add row to matchdata matrix with home team's data  
  matchdata <- rbind(matchdata, t(c(hometeam, awayteam, matchday, TRUE, scores[1],    
          possession[1], territory[1], otherstats[which(1:length(otherstats) %% 2 == 1)])))
  # Add row to matchdata matrix with away team's data
  matchdata <- rbind(matchdata, t(c(awayteam, hometeam, matchday, FALSE, scores[2], 
          possession[2], territory[2], otherstats[which(1:length(otherstats) %% 2 == 0)])))
}

# Create column headings
colnames(matchdata) <- c("Team", "Opponent", "Matchday", "Home", "Score", "Possession_Pct", 
                         "Territory_Pct", "Shots_On_Target", "Shots_Off_Target", "Shots_Blocked", 
                         "Passes_Completed", "Corners", "Effective_Tackles", "Clearances",
                         "Saves", "Offsides", "Fouls_Conceded", "Yellow_Cards", "Red_Cards")

# Change matrix to data.frame and change stat columns from character to numeric
matchdata <- as.data.frame(matchdata, stringsAsFactors = FALSE)
matchdata[6:7] <- sapply(matchdata[6:7], function(x) gsub("%", "", x))
matchdata[c(3, 5:19)] <- sapply(matchdata[c(3, 5:19)], as.numeric)

# Write to Excel
xlsx::write.xlsx2(matchdata, file = "matchdata2019_20.xlsx", row.names = FALSE)

sum(matchdata$Score[matchdata$Team == "Liverpool"])

## [1] 66

plot(matchdata$Passes_Completed[matchdata$Team == "Manchester City"], 
     type = "p", pch = 20, xlab = "Matchday", ylab = "Passes Completed")
lines(matchdata$Passes_Completed[matchdata$Team == "Manchester City"])

matchdata$Opponent[matchdata$Team == "Manchester City" & matchdata$Matchday == 19]

## [1] "Wolverhampton Wanderers"

totalfouls <- aggregate(Fouls_Conceded ~ Team, data = matchdata, FUN = sum)
totalfouls[order(totalfouls$Fouls_Conceded), ]

##                       Team Fouls_Conceded
## 10               Liverpool            243
## 3              Bournemouth            269
## 13        Newcastle United            276
## 14            Norwich City            278
## 11         Manchester City            283
## 4   Brighton & Hove Albion            288
## 6                  Chelsea            291
## 17       Tottenham Hotspur            296
## 19         West Ham United            301
## 15        Sheffield United            303
## 1                  Arsenal            307
## 9           Leicester City            309
## 20 Wolverhampton Wanderers            316
## 7           Crystal Palace            317
## 12       Manchester United            320
## 5                  Burnley            324
## 2              Aston Villa            327
## 8                  Everton            346
## 18                 Watford            350
## 16             Southampton            356

Conclusion

In this article we looked at how to do web scraping in R where the content we want to scrape is in static HTML format. In the next article we will look at how to scrape Javascript-rendered content from webpages, which is more advanced but can actually be easier because of the built-in tools in R for doing so.