Web Scraping

While most of the time we rely on programing language for data related tasks, sometimes it also comes handy to scrape data from various sources using packages such as Beautiful Soup or Selenium.

In my own research projects, I learned to use Beautiful Soup to scrape down historical paper information at AFA or WFA annual meetings. Recently, I also tried to maneuver a different package Selenium to scrape historical significant short position from various European government sites.

I don't claim to be expert on this field, but thought to share the code so that others can use these as a reference and stepping stone for their own project.