The Back to School Thread

Crone_sl said:

Hey bros, looking for maybe some suggestions. Been on the job hunt for about 3 months now, and haven't even gotten an interview. Now I realize this could be due to a number of different factors, but I feel maybe I need make a change, and was considering going to school for a certification, or at least something I could add to the resume. My time frame is between now and June 2015, so it fits pretty well with the spring semester somewhere.

I'm a nerd, so I feel I'm pretty tech savvy, however coding has always been boring. But I had a couple thoughts.

My 2 thoughts were either cramming for a CCNA cert between now and June. Most the community colleges around me have programs for it. Involves 14-18 credit hours, so that would be quite the cram while I'm working full time.

Other option I know far less about, was Big Data, or Hadoop. Coursera seems to have some stuff on it, and about as much as I know now, after looking for jobs for 3 months is that Hadoop skills seems to be in pretty high demand.

Now there is always the idea of saying F it, going back to school, finish an undergraduate, and going for a Masters in Big Data, Data Science, or some other business degree, but I'm not ready to take that leap just yet.

Any other suggestions or thoughts? I'm away from the wife and kids working until I get a job, or June 2015, whichever comes first, so I easily have the time to take night classes right now.

Click to expand...

You can learn Hadoop fairly easily. Or at least the concepts that drive Hadoop and how it manages Big Data. I imagine as long as you know how to write effective Map/Reduce jobs to clobber over HDFS files and HBase data stores you can at least throw it on the resume. But just saying "Hadoop" anymore is very vague since the ecosystem is massive and grows on a daily basis it seems with new Apache projects. And like I said I could teach someone how to write MapReduce in an afternoon but that doesn't mean they fully grasp the underlying concepts, the tradeoff between NoSQL and RDBMS, effective table/key structure or anything like that. Then of course they wouldn't know things like Pig/Hive/Sqoop/Impala/Flume or any of the other widely used add-ons.

-edit-

And I don't know if I'd recommend getting a masters or anything unless you're going for a data architect job at like Facebook/Twitter/Google or something more academic. I'm sure most places hiring someone for Hadoop just are concerned with you being able to stand up an arbitrarily large cluster (or spin up an AWS) and install HBase on it and perform jobs against HDFS files thrown up there or data ingested into HBase

Tenks_sl said:

You can learn Hadoop fairly easily. Or at least the concepts that drive Hadoop and how it manages Big Data. I imagine as long as you know how to write effective Map/Reduce jobs to clobber over HDFS files and HBase data stores you can at least throw it on the resume. But just saying "Hadoop" anymore is very vague since the ecosystem is massive and grows on a daily basis it seems with new Apache projects. And like I said I could teach someone how to write MapReduce in an afternoon but that doesn't mean they fully grasp the underlying concepts, the tradeoff between NoSQL and RDBMS, effective table/key structure or anything like that. Then of course they wouldn't know things like Pig/Hive/Sqoop/Impala/Flume or any of the other widely used add-ons.

-edit-

And I don't know if I'd recommend getting a masters or anything unless you're going for a data architect job at like Facebook/Twitter/Google or something more academic. I'm sure most places hiring someone for Hadoop just are concerned with you being able to stand up an arbitrarily large cluster (or spin up an AWS) and install HBase on it and perform jobs against HDFS files thrown up there or data ingested into HBase

Click to expand...

You make it sound so easy Tenks. haha! Unfortunately I don't know anything about Hadoop, or how to write Map/Reduce jobs. In my brief research, I noticed quite a few graduate programs, that are getting more popular that focus on data science/architect, which is why I mentioned the masters. But that's pretty far outside my reach right now.

At this point, from a return on investment stand point, we have a Cisco certified lab + prep courses at a community college that I'm going to enroll in, and then get my CCNA. Friend went through it and apparently it was a pretty impressive lab for lots of hands on experience.

MapReduce at a fundamental level is very, very easy. Just some off-the-bat terms

* HDFS - The underlying filesystem Hadoop uses to manage very large files. To achieve parallelism a single logical file is split up into many (and redundant) physical files spread throughout the cluster. So even though you have a 10 TB file called MyFile.txt it is actually thousands of smaller files spread around the cluster.
* Driver - This is what launches your MapReduce and sets a bunch of config to the Job.
* Map - The first pass phase. The goal of Map is to find the commonality you are hoping to link all records together.
* Reduce - The second pass phase. the goal of Reduce is to do business rules against the linked data.

So basically the normal flow you have is: Driver --> Map --> Reduce. [There are more "hidden" phases but these are the major ones]

So for your Mapper you are simply trying to find what joins a certain set of records together. If you want you can think of this almost as a foreign key relationship or a JOIN clause in SQL. Which is funny because this is exactly what some add-ons like Hive do is convert SQL statements into canned MapReduce jobs. Because it really is that simple.

Lets say our record looks like this: ${FirstName}::${LastName}::${Salary}

So you get a record one-at-a-time to your Mapper. The number of Mappers launched is determined by the size of the file you're reading (by default that is 1 mapper per block.) You read the record. We know fields are delimited by double colon so we split the record on that. The goal of this MapReduce is to find the average salary of all people with the same last name. Cool. So our mapper will write to the reducer KEY=LastName VALUE=Salary. So our mapper would be like: (this is all just handwritten psuedo-ish code so may have some syntax issues)

Great so our mapper is outputting all people grouped by last name and their salary. So now we need our reducer to get all this information and calculate the average. Also pretty easy:

And there we go. If we had access to all Americans names and salaries we could have a huge line delimited text file and we could output how much Smith makes on average. Like I said as a concept it is pretty simple stuff nothing too magical. The actual implementation and how you attack certain MapReduces which are less straight forward than this is where it gets a bit tricky.

Thanks Tenks for the over view. That all made sense to me, and I'm sure writing the code portion of it would come with experience. Big Data in theory/concept sounds much more interesting to me than networking (CCNA), but I didn't really see any "Hadoop certs", and so it didn't look as easy to get into that business without going through graduate school.

I'll just link your post on my resume when I say Hadoop Master, cool? lol

There are certifications. Look at Cloudera. They offer their Cloudera certs I believe. Cloudera is basically a middleman distro of Hadoop. Basically takes the ecosystem, make sure it is configured and plays nice together and ships it out then makes their money off education and certification. If it wasn't for Cloudera Hadoop would be a massive pain in the ass to stand up. It is nice downloading their distro and having the guarantee the Hadoop/HBase/Hive/Pig/etc/etc all play nice together.

I liked math a lot until I got into weird classes. Numerical Analysis was brutal

Noodlemod_sl said:

I liked math a lot until I got into weird classes. Numerical Analysis was brutal

Click to expand...

Fuck that class. Im taking a Numerical Analysis class based around nuclear engineering. All we do is use Python to code shit, and it sucks.

They could have you just doing it by hand all day with no tests and your grade is based on homework that solutions for do not exist in any book or solution manual because the teacher wrote the book and did not write a solution manual and also it is his last semester teaching before retirement so he don't give a fuck.

I suppose I don't really have it that bad in comparison haha.

I'm just not too fond of coding I've found, it's tedious to me.

My girlfriend is preparing to take some kind of general administrative assistant exam or something, and it involves doing some math. She showed me a couple of the sample questions, which consist of word problems like "If it takes 3 typists 24 hours to type up a report, how long will it take 2 typists to type up 2 reports?" (I'm probably slightly mistaken in the specifics, but it was along those lines).

That shit blew my mind, and even after seeing the answer I don't fully understand how/why they set up the equation to solve it. In short, fuck math! Humanities4life!

McCheese_sl said:

My girlfriend is preparing to take some kind of general administrative assistant exam or something, and it involves doing some math. She showed me a couple of the sample questions, which consist of word problems like "If it takes 3 typists 24 hours to type up a report, how long will it take 2 typists to type up 2 reports?" (I'm probably slightly mistaken in the specifics, but it was along those lines).

That shit blew my mind, and even after seeing the answer I don't fully understand how/why they set up the equation to solve it. In short, fuck math! Humanities4life!

Click to expand...

wormie_sl said:

That shit aint math.

Click to expand...

1r = 3 * 24h = 72h
2r = 2 * x = 144h <=> x = 72h
So it takes 2 typists 72h to type 2 reports. In other words, double the workload with 2/3 the manpower is triple the time needed.

wormie_sl said:

When someone say they enjoy math, for example Troll a few posts above, they arent talking about arithmetic. That shit aint math.

Click to expand...

Yet it is math. I realize you're the theoretical math guy (I think I remember from before), but I too loved math.

I even loved it through most of my degree. Then I took engineering math, numerical analysis, and discrete math and I said "fuck this, I hate math now"

I'm actually only a few courses away from a BS in Mathematics, but I'll never finish it. Engineering just had so many math classes that I got that close.