Todo List

Namespace ConScrap

save report to file

save to csv or faunadb

improved cli interface

exe with working env vars

scrap for only new reports and/or html format

implement discord module simple functions that map YahooComments to Data points

Member ConScrap.Browser.GetAllEntries (string ticker="PKK.CN")

figure out how to show replies

Class ConScrap.ConScrap

get list of stocks

Member ConScrap.ConScrap.FetchStocks (string[] stocks, bool sendDiscord=true, string dataPath="data")

rate limit selenium instances to 10 see bser logic

skip failure tickers

Member ConScrap.ConScrap.ProcessStock (string stock, Types.FetchConfig fetchConfig)

batch comments in 10 to send off

Member ConScrap.ConScrap.twitFetchStocks (bool sendDiscord=true, string dataFolder="data/twits")

rate limit selenium instances to 10 see bser logic

skip failure tickers

Member ConScrap.Parse.ExtractYahooConversationsHtml (string yahooHtml)

Mark urls in tex using \url{} after removing characters

Member ConScrap.Parse.GetYahooComment (HtmlAgilityPack.HtmlNode commentNode, bool parseForTex=false)

ignore yahoo comment replies for now

Class ConScrap.StockTwitsClient

label this package as v1 (intend to support v2 as a hobby)

Member ConScrap.StockTwitsClient.StockTwitsClient (int rateLimit=200)

TODO figure out how to handle urls when v2 comes out