BI Dev/Data Analyst/Data Science and Careers (Official Discussion Thread)

[Macallik86]

That's and window functions are the two things in SQL, I haven't mastered yet. But with respect to FOR XML/STUFF, you know that SQL Server 2016 released some new string functions. They should help ease string manipulation. And I just found out there's a FOR JSON too. But I haven't used it at all.

FWIW I created flashcards on the stuff (no pun intended) I learned, but I am having trouble retaining the info I learned since I don't require that type of complexity in my queries very often. I do have a Google Doc w/ the key notes, so I will reference that if an opportunity provides itself.

On a related note, I'm having the complete opposite experience with Pandas (via Python). It's to the point where I have replaced the step of copy/pasting the data into Excel -> saving a CSV and loading into JupyterLab.

I was lit at an office party yesterday telling my boss' boss that we should probably try and pivot all the new hires to get more proficient in Pandas over Excel. Excel is a powerful tool, but in terms of replicating results, it's a shytshow. There have been plenty of times when I'd do +2 hours of hard work to reach the right answer to a data question, but it ends up being incorrect because of user-error like a formulae not going all the way down, the workbook calculations were set to manually update, I pasted the data off by one column inadvertently, etc.

With Pandas, it is much easier to get summary data, and more importantly, to explore the data and tease out more in-depth stories. And I say this as someone who is only like a third of the way through a course on Pandas. Even if you are a wizard in Excel, I strongly encourage you to get your Python/Pandas game up to do even more w/ less work

 

[trap101-ETHout-Allegri-In]

How I not see this thread before. Working on Stats and Math undergrad then masters. Im in for this....

 

[Rawtid]

 

[Secure Da Bag]

If you're a data scientist and can pivot to ML Engineering....please do so cuz they money these tech companies are paying is absolutely bonkers

 

[greenvale]

Can't stress it enough, if you're in this field you should know SQL. Like actually know it. It's wild how many people I work with who "know SQL" who don't know shyt at all. Making me work twice as hard for their fukk ups

 

[MikelArteta]

FWIW I created flashcards on the stuff (no pun intended) I learned, but I am having trouble retaining the info I learned since I don't require that type of complexity in my queries very often. I do have a Google Doc w/ the key notes, so I will reference that if an opportunity provides itself.

On a related note, I'm having the complete opposite experience with Pandas (via Python). It's to the point where I have replaced the step of copy/pasting the data into Excel -> saving a CSV and loading into JupyterLab.

I was lit at an office party yesterday telling my boss' boss that we should probably try and pivot all the new hires to get more proficient in Pandas over Excel. Excel is a powerful tool, but in terms of replicating results, it's a shytshow. There have been plenty of times when I'd do +2 hours of hard work to reach the right answer to a data question, but it ends up being incorrect because of user-error like a formulae not going all the way down, the workbook calculations were set to manually update, I pasted the data off by one column inadvertently, etc.

With Pandas, it is much easier to get summary data, and more importantly, to explore the data and tease out more in-depth stories. And I say this as someone who is only like a third of the way through a course on Pandas. Even if you are a wizard in Excel, I strongly encourage you to get your Python/Pandas game up to do even more w/ less work

Or if you use macros easy. Stuff that used to take days I automate excel with macros to take five min

 

[trap101-ETHout-Allegri-In]

IDK if this has been posted:

About this Book — The Data Science Interview Book

Preparing for interviews is a stressful task. There is an enormous amount of resources available in the internet, multiple repositories, there are even companies that help students prepare for interviews at the Big Tech companies. The idea here is to create an accessible version of these...

dipranjan.github.io

 

[Rawtid]

 

[Secure Da Bag]

When non-sql devs make databases and you have to normalize them, it makes me wanna take a sledgehammer to everyone involved. Something that should take 2 or 3 weeks, may take 2 or 3 months.

"This is why I get paid the big bucks. This is why I get paid the big bucks. This is why I get paid the big bucks...." Hopefully that lie sticks.

 

[Gritsngravy]

What are the credentials someone would have to be hired as a data scientist/engineer?

 

[greenvale]

What are the credentials someone would have to be hired as a data scientist/engineer?

Very barebones but

Data Scientist:
Likely a lot of work and understanding of various models. Logistic Regression, Decision Trees, Random Forest, etc.
Python/R Knowledge
Strong presentation/communication skills
Data cleaning skills - many don't actually know this so this is a way to stand out

Data Engineer:
SQL - more than just selecting and joins... think stored procedures, query optimization, normal forms for db design, parameterized quality checks, etc.
Knowledge of one of the bigger database platforms such as MS SQL
Knowledge of some ETL tool such as SSIS, hadoop, pyspark
Python knowledge is a nice to have
Requirements gathering skills

Both are buzzing right now, but ultimately you can't do data science without great data engineers building the pipelines for your data.

 

[Gritsngravy]

Very barebones but

Data Scientist:
Likely a lot of work and understanding of various models. Logistic Regression, Decision Trees, Random Forest, etc.
Python/R Knowledge
Strong presentation/communication skills
Data cleaning skills - many don't actually know this so this is a way to stand out

Data Engineer:
SQL - more than just selecting and joins... think stored procedures, query optimization, normal forms for db design, parameterized quality checks, etc.
Knowledge of one of the bigger database platforms such as MS SQL
Knowledge of some ETL tool such as SSIS, hadoop, pyspark
Python knowledge is a nice to have
Requirements gathering skills

Both are buzzing right now, but ultimately you can't do data science without great data engineers building the pipelines for your data.

I was speaking more so on if you need a degree or could you get a job only with certs
And if you get a degree what degree matters most

 

[greenvale]

I was speaking more so on if you need a degree or could you get a job only with certs
And if you get a degree what degree matters most

Any business/comp-sci/analytics degree is fine. The key really is your experience. Unless it's entry level you will likely have to do a technical interview so that's where you prove your worth imo

 

[Gritsngravy]

Any business/comp-sci/analytics degree is fine. The key really is your experience. Unless it's entry level you will likely have to do a technical interview so that's where you prove your worth imo

Well I got an electrical engineering degree, I plan getting a masters in another year or so, but I was curious about maybe get into cloud or data to do on the side in the future versus my main career path, so that’s why I wanted to see what jobs will be looking for
Is it possible to potentially do data as secondary job? Like maybe some wfh job I could do while doing a 9-5?

 

[Conan]

Data Engineering is where it's at.

In data science but can't do shyt without good data flows.