So You Want To Be A SysAdmin

Hopefully people looking for this information will find it here; one of the most stifling things about the IT profession is, to me, the total lack of transparency on behalf of IT professionals regarding their development and their skills.  Unfortunately, too often they don’t want to reveal this information because it is so sorely lacking.

So that’s the first and most important point I can make to any aspiring IT professional: don’t be one of those IT professionals!  Take your professionalism seriously.  If you call yourself an engineer, you’d best be certified by organizations relevant to your field and engaged in planning, delivering, and operating complex software and hardware solutions.  Same goes for administrators, though the planning and delivering will be greatly lessened or non-existent.

Though I disagree with  shots fired at the use of the term “engineer” in the world of software, I sympathize with a position against its overuse.  I myself have distinguished between “administrator” and “engineer” on this blog (somewhere around here, I thought…).  I respect the division roughly outlined by comparing industry standards such as RHCE and RHCSA or LFCE and LFCS requirements.  It is on these grounds that I find my description at the end of the previous paragraph to be generally accurate.

And despite the article’s critique, it seems to me these terms are being used well.  The article follows its request that “programmers” stop calling themselves engineers by saying “it undermines a long tradition of designing and building infrastructure in the public interest.”  This strikes me as a strange and ironic thing to say; just what does the author of this article consider free and open source software to be?  Is it not infrastructure in the public interest which is designed and built by “programmers”?

The article continues:

Traditional engineers are regulated, certified, and subject to apprenticeship and continuing education. Engineering claims an explicit responsibility to public safety and reliability, even if it doesn’t always deliver.

The title “engineer” is cheapened by the tech industry.

For one thing, IT professionals are often subject to many regulations, particularly if working in fields such as health care or government sectors.  For another, there are many high quality and well-respected certification tracks offered for IT professionals.  Many of these certifications require continuing education and verification of professional practice.

So maybe he’s only concerned at the fact that these qualifications are not required of IT professionals, and therefore their titles may be misleading?  If that is true (and it is), then it must be rectified, no?  Rather than disregard the extreme importance of the work being done in free and open source software and proprietary solutions for the same fundamental infrastructure problems (encryption, networking, file systems, logical volume management, kernel software, etc.) and deny appropriate titles of those who do that valuable work, perhaps we should focus on the problem of unqualified IT professionals acting as such.

For I agree that this problem exists to a very unacceptable degree.  There are many, many high-level IT professionals (e.g. CIO, management) and many low-level IT professionals (engineers, administrators, support staff) who fail in their obligations to their subject matter.  In fact, it’s such a problem that IT professionals are starting to become more and more open about suggesting that it’s okay and pretty common not to be that learned or that good at one’s job.  It’s sort of trendy right now among IT professionals to have big sort-of coming-out moments where they identify themselves as those who suffer from “impostor syndrome.”   Where it isn’t just a whine about a lack of self-esteem, it’s a coy way by which to claim superiority and yet the humble agony of thinking otherwise.

All silliness, of course.  But, again, I sympathize; the rapidly expanding domain of information technology and the growing realization that it’s actually really hard to be really good is starting to weigh down on people.  With FOSS, especially, you have to keep on top of things at a pace that can seem downright impossible without devoting countless hours of one’s life.  You have to subscribe to a lot of RSS feeds, run a lot of virtual machines in a lab, and build and support a lot of various software solutions across a very large problem domain in order to be competent enough to deserve the title of “system administrator,” much less “systems engineer.”

But that doesn’t mean we should curl up in the fetal position and pretend that it’s not really that hard, or that those who succeed aren’t really that awesome (for though some aren’t, some very much are, and I’d suggest we should aspire to be the latter), or that we aren’t capable of rising to this challenge.  The middle way here is to recognize the difficulty and begin to develop serious criteria for professionalism and adequate means by which the greatest amount of aspiring IT professionals are capable of meeting them.  We need transparency, open education and training, and a meritocracy.  That’s all.

But training programs are often downright terrible.  Borderline useless.  I’ve sat through classes where we basically just walk through the book, and since the book is often many hundreds of pages, this simply consumes 80% of class time, leaving about 20% for narrow, nearly-pointless lab exercises. Other times, training programs are great, but you really need to take it seriously, read the material once before class, and be ready to hit the labs hard, designing your own problems if those offered by the course are inadequate.

One major problem in information technology is that it is an increasingly difficult job to master which is sought out by a very, very large skill range.  This isn’t like being a doctor where it’s a very difficult job to master and low-skill folks are filtered out before they even make it out of school.  This is like being a doctor (with thankfully less risk of death, but fundamentally very similar otherwise in many important ways) in an environment where medical school is optional.

It has continued to surprise me that there are few system administration or engineering concentrations in universities with which I am acquainted.  The knowledge required is much different from that which is offered in typical computer science tracks, though that information is often critical and almost always helpful, as well.

But, all that said, one of the greatest things about the IT industry right now is the growing availability of and respect for high-quality certification programs.  Sure, some of them suck, as mentioned above, but the Red Hat examinations, the Microsoft examinations, and the Linux Foundation examinations appear to me to be of high quality, and those who genuinely pass them without cheating* are required to learn quite a lot.  I received multiple MCITP certifications and I worked for years, reading approximately ten thousand pages of IT texts in order to pass them.  The knowledge has proven invaluable to my career, and I can testify on their behalf.

Finally, not only can we learn how we ought to be from the noble field of engineering, we can also learn how we ought not to be.  We don’t want IT to become beholden to hundred-thousand dollar education and certification programs.  And we don’t want work experience to count for so much that it prevents intelligent, hard workers from rising through the meritocracy.  We should keep that in mind as the future unfolds.

In light of all this, I thought I’d cross-post this from an question I answered.  If you’re still reading this, and you’re interested in a practicing software systems engineer’s perspective on what it takes to go from interested home user to professional, here you go:

Probably the most important aspect of system administration off the bat is the init system, and since Fedora 22 uses systemd and CentOS/RHEL 7 make use of it as well, you’ll be well-acquainted there. CentOS/RHEL 6 and before make use of upstart, so you may be a bit out of practice if you’re not using systemd’s compatibility features and you only make use of Fedora – it adopted systemd back in mid 2011 with Fedora 15. The second most important aspect is probably package management, and since Fedora made use of yum up until 22, you’re probably well studied on that front (CentOS/RHEL 5, 6, and 7 use yum).

Almost all of the other software available for Fedora is available for CentOS/RHEL, albeit often in the form of a previous feature version which is maintained by Red Hat engineers with security patches (often backported from future releases of the software). You’ll certainly be familiar with the most commonly used software, but you may have to tolerate the absence of some of your preferred features. If you grew with the software, you should still be capable of functioning, and if you know the software well, you’ll be a great administrator of it at previous feature levels.

But, as you likely suspect, home system administration is most often very different from professional system administration. The most important distinction between professional administration/engineering and personal use is probably that you likely don’t frequently make use of software which is most often used in enterprise environments. When you do, you almost certainly don’t make use of it at a similar level of complexity or fullness. You probably, for instance, don’t do a lot of postfix or sendmail configuration at home. You might not make much use of LVM or do much file system administration. Performance monitoring and reporting with something like Performance Co-Pilot, multiple system administration with a solution such as Red Hat Satellite / Spacewalk, managing large KVM solutions, etc. are things that you may not do at home at all. Some of the activities which you are more likely to have engaged (e.g.auditd, cron, sar, various backup solutions) are going to be implemented through many features you may never have touched or even thought about.

But you know what? You can find out exactly how well you fare if you simply whip up some KVM guest domains on your Fedora rig and install CentOS. Give both major versions 6 and 7 a shot. CentOS and RHEL of the same major and minor versions are virtually indistinguishable from one another in practice (they are binary-equivalent, but you have to learn some of the RHEL-specific utilities such as subscription-manager, but it’s typically trivial stuff).

The most wonderful thing about being an aspiring system administrator / engineer today is the extremely high quality and availability of sophisticated open source software used in professional environments. You can download and run multiple enterprise-grade operating systems, hypervisors, and innumerable applications, all on your home desktop. You can be using on your home computer the exact same software used in health care, government, and scientific sectors. It is incredible – such an opportunity was not available to those of us who grew up in the days of Windows-or-Ph.D-level-computer-science-comprehension if you expected to do anything remotely complicated or relevant to your friends (i.e. video games).

For a good objective measure of your capabilities, I’d recommend that you check out certification programs such as the Red Hat Certified System Administrator or Linux Foundation Certified Administrator. You don’t have to take them or the offered courses, even; create for yourself a virtual lab environment and see if you can meet the requirements for the examination. If you have the time and the interest, you have all the tools needed to create meaningful laboratory replicas of enterprise environments. Think up your own basic test questions which would be asked of someone in the examinations (a learning process in and of itself if you don’t know what to ask). Build some working server/client relationships with common software platforms and infrastructure (KVM/QEMU/libvirt, postfix, OpenLDAP, BIND, FreeIPA, Apache httpd, etc.) and challenge yourself to accomplish typical system administration tasks. Join boards such as Ask Fedora (or forums for CentOS) and start solving people’s problems – it’ll keep you well-rounded and constantly evolving in your capabilities (something which is hard to provoke in oneself on occasion), and you’ll be participating directly and valuably in the development and stewardship of the software by which your livelihood will be earned.

If you have been an interested power user of Fedora for many years and you are familiar with the structure of the operating system at a pretty deep level, and you already have fairly sophisticated knowledge of multiple shells, text editors, and perhaps scripting languages, the biggest hurdle you have to jump is knowledge of that software set.

In fairness, you can get away with knowing a lot less than is required by those examinations if you apply to a position where they are willing to teach you or their hiring practices are inadequate (lol), but you don’t want to put yourself in that situation unless you’re up front about your ability level and confident that you can really get up to speed quickly. Being a n00b in the face of a complex IT disaster is not desirable.

For what it’s worth, I was a lifelong nerd (but only used Microsoft products until college) and I progressed in my career by working at two different help desks (departmental ones that aren’t call centers, but do Tier 1 through 3 support in house are less common, but so vastly superior in terms of the experience and interest they hold for employees when compared to call centers that it’s hard to put them in the same league) and then moved up to system administration, then engineering, all over the course of about 7 years, so that’s at least one small set of relevant data points for you.

P.S. I have a pretty bitchin’ home server sysplex if I do say so myself, and I use a Fedora 22 Workstation as my primary system for work and home use. If you’re interested in the kind of challenge a practicing GNU/Linux system engineer might take on at home, see here for details. I also run a virtual lab environment on my Fedora 22 Workstation rig with the KVM/QEMU/libvirt stack wherein I have fresh installations of all the major Linux distributions that I can use (in conjunction with snapshots) to recreate problems and test theories and solutions. So, Fedora 22 is a huge part of the development of my system administration and engineering skills.

*Cheating is, sadly, all too common in the case of Microsoft examinations; hopefully less so in the Red Hat and Linux Foundation examinations given their nature as proctored, live exercises and not answers to multiple-choice questions.  Though I really thought Microsoft’s multiple choice exams were well-designed and brutal, one can hardly argue that they aren’t more exploitable by cheaters.

This entry was posted in Academics, Information Technology and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s