Quoting this for others to see.
Software Development and Programming Careers (Official Discussion Thread)
- Thread starter Blackking
- Start date
-
- Tags
- careers programming
Thanks didn't think anyone read/listened
like in real life.Why?well, depends on what schools
berkeley basically had 3 'intro' courses for cs engineers that were mandatory, each a semester each
first one was scheme, basically super abstract and lots of lambda expressions
second one was mostly what I wrote, c+assembly+ hardware structure. I remember one project (two weeks to do) was basically to emulate nix directory and file structure in cthat was FRESHMAN year lol
third one was OOP and java, I APed out of that thankfully- still remember a facebook group picture of huge cup of coffee the professor was drowning everyone in
all 3 were 'weeder' courses. so top schools actually want to weed out people who arent good enough. so the scope of programming interest is quite important to know where to start.
I know a lot of people who got into programming because they thought it was cool and they liked being computer users and pseudo hacking with kiddie scripts. they later regretted that decision dearly.
and yeah, java fukking sucks. any server built on java isstatus
TrebleMan
Superstar
Getting this post back to the top. Need a break from the politics.
Learning coding with Haskell has been going well.
About to write a web scraper with it, and was wondering about the implications. I wanted to hit a RESTful api to start getting some NBA stats, but was curious what is the difference between that and trying to scrape a site like pro-basketball reference instead.
I'm planning to try and figure out how certain stats correlate with wins and the margin using a large sample size with a ton of game logs like the ones provided on those sites. Only thing is it'll be a lot of queries. I could probably pop all of the scraped info to a DB so I won't have to keep querying their site, except I'll have to at least once a day for newer games. Are there better ways than this? Should I just cough up for a stats.nba.com api key?
Learning coding with Haskell has been going well.
About to write a web scraper with it, and was wondering about the implications. I wanted to hit a RESTful api to start getting some NBA stats, but was curious what is the difference between that and trying to scrape a site like pro-basketball reference instead.
I'm planning to try and figure out how certain stats correlate with wins and the margin using a large sample size with a ton of game logs like the ones provided on those sites. Only thing is it'll be a lot of queries. I could probably pop all of the scraped info to a DB so I won't have to keep querying their site, except I'll have to at least once a day for newer games. Are there better ways than this? Should I just cough up for a stats.nba.com api key?
KritNC
All Star
So I have spent the past 3 months building something similar to what you are talking about. I basically scrape 3 different sites to get statistics on NBA, NCAA Football, and NFL games then grab data on public betting percentages and create a score to predict the correct team to bet.
You can check it out here:
CheckTheLines
What I have learned is that web scraping is a bytch. It is very fragile and takes a lot of testing/time to ensure you are getting the correct data. Also you are at mercy of whoever runs the site. If they switch around their DOM at all you will have to rewrite your code. I would recommend going the API route if you can find one that works for you.
As far as the last part, you for sure should be storing this data. After I scrape my data I immediately store it in postgres so I can use it later without having to make another HTTP request to grab it.
TrebleMan
Superstar
Great reply, just what I was looking for, thanks breh. I'll definitely go the API route, just need to find some that will give the info I'm looking for.
Actually your app was what I had in mind, but couldn't remember where I saw it. That's really a great job and the results speak for themselves. Congrats!
KritNC
All Star
No problem, you shoud check this out MySportsFeeds - NFL Scores & Schedule
It is free and has a live API. Once I get some more time I think I might rewrite a portion of mine to grab data from that API.