I’m presenting today at DevDataDay in Birmingham.¬† First time in a while for me, but I’m excited about the opportunity to go to sessions again and learn something (more on that in a bit).¬† I realize this blog has been a bit dusty for a while, but I’m gonna try to do better about that.
TOPIC:¬†¬† Theory of Cloud Database Administration
SPEAKER: Stuart Ainsworth
ABSTRACT:¬†The cloud is more than just a marketing term; it’s a model for designing scalable, distributed systems and assigning ownership to various components of those systems.¬† Data is a crucial part of that model, and there are lots of challenges ahead.¬† This presentation will explore the history of cloud computing, the current state of data in the cloud (using SQL Server 2016), and the impact on career choices for data people.
I’ve been rereading the book Slack: Getting Past Burnout, Busywork, and the Myth of Total Efficiency recently.¬† As I alluded to in my last post, my life has been rough for the last few months.¬† My nephew’s passing took the wind out of an already saggy sail; I’ve spent a great deal of time just trying to balance work, family, and life in general.¬† Some people turn to counselors; I turn to project management books.
The premise of the book is that change requires free time, and that free time (slack) is the natural enemy of efficiency.¬† This is a good thing; if you are 100% efficient, you have no room to affect change.¬† Zero change means zero growth.¬† I’ve been a proponent of slack for a while (less successfully than I’d like); it makes sense to allow people some down time to grow.¬† Just to be clear, slack isn’t wasted time; it’s an investment in growth.¬† Slack tasks include:
- Research into interesting projects.¬† Lab work allows you to experience the unexpected, which gives you time to prepare for the unexpected in production
- Building relationships. Teams are built on trust, and trust is earned through building relationships.¬† Teams that like each other are more likely to be successful when it comes to problem solving.
- Shadow training.¬† Allow team members to work in other teams for a while; learn how the rest of the company operates.
In short, slack is necessary in order to promote growth; if you want your organization to stay ahead of it’s competition, cutting resources in the name of efficiency is sure-fire plan for losing.¬† The best advice for slack time is the 80/20 rule; run your team at 80% capacity, and leave 20% for slack.¬† In the case of emergency, slack time can be temporarily alleviated, but it’s the responsibility of management to return to normal work levels as soon as possible.
So what does this mean for me personally?¬† In the name of efficiency, I let slack time go.¬† I work a full time job, a couple of different consulting gigs, act as a chapter leader for AtlantaMDF, and am an active father.¬† I have no hobbies, and suck at exercise.¬† I love to travel, but trips are planning exercises in and of themselves.¬† In short, I have zero slack to deal with emergencies.¬† When something goes wrong and time gets compromised, I immediately feel guilty because I’ve robbed Peter to pay Paul in terms of time.¬† That’s not living,
I’m done with that.
Change is incremental, so I’m not planning on upsetting the apple cart just yet, but I am trying to figure out ways to make my slack time more of a recharge time.¬† Don’t get me wrong; I waste time.¬† I sit and stare at Facebook like the rest of the modern world; I binge on Netflix when new series drop.¬† That’s not slack, and it doesn’t recharge me. Slack is using free time to grow, to change.¬† My goal is to find an hour a week for growth-promoting free time.¬† I’ll let you know how I’m doing.
I’ve been reluctant to write this post for a couple of months now, in part because I don’t usually use this forum for posting personal stuff, and in part because I didn’t know what to say.¬† I’ve got another post in mind, however, and I felt like I needed to give this topic the respect it needed before I could pick up where I left off.¬† Bear with me.
On December 1, 2015, my nephew took his own life.
No matter how many ways I tried to write that previous sentence, none of them seemed to adequately capture what I’m thinking or want to say, so I’ll leave it alone and try to finish this post.¬† Cris was a smart, funny, kind young man with whom I didn’t spend near enough time as he grew up; I really only got to know him over Facebook as an adult, and I loved him.
He had battled depression for years, and he lost the war.¬† He left behind a grieving fiance, a set of in-laws who loved him very much, his mother (my sister),¬† grandparents, great-aunts, great-uncles; all of us were shocked, and I didn’t realize how much I would miss him.
Family is important, y’all.¬† Don’t miss out, and don’t let anything keep you from loving yours.
I’ve been presenting a lot on Big Data (specifically Hadoop) from the perspective of a SQL Server DBA, and I’ve made a couple of recent observations. ¬†I think most people are aware of the fact that data generation is growing at a staggering rate, with some estimates as high as 44 zettabytes by the year 2020; what I think is lacking in the SQL Server community is a rapid movement among database professionals to expand their skills to highly scalable Big Data platforms (like Hadoop) or streaming technologies. ¬†Don’t get me wrong; I think there’s people out there who have made the transition (like Michelle Ufford; SQLFool, now Hadoopsie), and are willing to share their knowledge, but by and large, I think most SQL Server professionals are accustomed to working with our precious relational system.
Why is that? ¬†I think it boils down to three reasons:
- ¬†The SQL Server platform is a complex product, with ever increasing opportunities to learn something new. ¬†SQL 2016 is about to drop, and it’s a BIG release; I expect most SQL Server people to wrap themselves up in new features and learn something new soon. ¬†There’s always going to be a need for deep expertise, and as the product continues to mature and grow, it requires deeper knowledge.
- Big Data tools are vast, untamed, and very organic. ¬†Those of us accustomed to the Microsoft development cycle are used to having a single official product drop every couple of years; Big Data tools (like Hadoop) are open-source, prone to various forks, and very rapidly developed. ¬†It’s like drinking from a firehose.
- It’s not quite clear how it all fits together. ¬†We know that Microsoft has presented some interesting data technologies as of late, but it’s not quite clear how the pieces all work together; should SQL Server pros learn Azure, HDInsight, Hadoop? ¬†What’s this about U-SQL? ¬†StreamInsight, Spark, Cortana Analytics?
The first two reasons aren’t easily solved; they require a willingness to learn and a commitment to study (both of which are difficult resources to commit). ¬†The third issue, however, can be easily addressed by the following graphic.
This is Microsoft’s generic vision of a complete end-to-end analytics platform; for the data professional, it’s a roadmap of skills to learn. ¬†Note that relational engines (and their BI cousins) remain a part of the vision, but they’re only small pieces¬†in an ever-increasing ecosystem of database tools.
So here’s the question for you; what should SQL Server people do about it? ¬†Do we continue to focus on a very specific tool set, or do we push ourselves (and each other) to learn more about the broader opportunities? ¬†Either choice is equally valid, but even if you choose to become an expert on a single platform in lieu of transitioning to something new, you should¬†understand how other tools interact with the relational system.
What are you going to learn today?
A couple of weeks ago, I briefly mentioned the Karma Go, and how I intended to use it to stay connected at the 2015 PASS Summit in Seattle. ¬†The good news is that the little fella worked great inside the convention center; the better news was that the convention center had upgraded its WIFI capabilities, so my Go was unnecessary. ¬†However, something else has recently happened that may be helpful to people closer to home;¬†Karma has recently announced that a monthly unlimited plan is now available for the Go. ¬†Speed’s capped at 5MB, which isn’t super-fast – unless you live in Jackson County, Georgia.
You see, here in Jackson County, we have only one option for broadband: Windstream DSL. ¬†When it works (as it has for me), it’s fine; I have the 12MB package, which is OK for working from home. ¬† If I have an outage, I fall back to my Go and I’m covered (that’s happened twice since I got the device in August). However, a lot of people in Jackson County pay for a service that is unreliable (to say nicely); quotes pulled from a Facebook group dedicated to Windstream issues in the county:
“Really tired of the constant outages. I’ll reset and the Internet comes back in for 2 minutes then goes right back out. It’s just awful!”
Or sometimes it works, but at speeds far less than what you’d expect
“Service today at the office [in] Pendergrass…. [(0.57 MB down/.42 MB up)] makes life interesting trying to get business done… Thank God my Ipad is ATT. “
Or sometimes they don’t show up at all
“I moved. Windstream said they’d be here today. I called at 4:30 to ask when since they haven’t called. The service rep said we were next in que and it is now 7:30. No show. How typical. Lies and the runaround. Windstream is beyond incompetent.”
Windstream was hit with a $600,000 fine by the Georgia Governor’s Office of Consumer Protection, and while the service has improved for some, many of my neighbors are still being overcharged for a service they’re not receiving. Some folks in the group had read about the new unlimited plan for Karma Go and wondered if it could be used to replace Windstream. The blog announcement for Neverstop includes this caveat:
“Can I ditch my home internet provider?
Neverstop isn’t meant to be a replacement for your home internet (yet). It’s a way to have internet anywhere, anytime. Speeds on Neverstop aren’t fast enough to be comparable to a wired home internet connection. It’s not practical for most people to use Neverstop as their only internet connection, but if you’re a light user and looking to cut down on costs, you’re welcome to give it a try. Lots of us at the Karma office are already using it as a replacement for a phone plan. Who needs minutes and texts these days?”
I thought I would help people decide for themselves; I asked the Facebook group for general locations in Jackson County where people were interested, and I headed out.
Here’s my lessons learned:
Jackson County, Georgia is bigger than I thought.
I had originally allocated an hour for my trip, but it took nearly 2 (and the starting point above is not near my house). The area is semi-rural; there’s lots of little suburban areas and towns surrounded by farms and ranches. I-85 runs through it, so cell coverage (even Sprint) is pretty good. I streamed music over the Go the entire trip, and had no hiccups; everywhere I stopped (and whenever I glanced at it), I had at least 2 bars of signal, and usually 3. In hindsight, I probably should have streamed video as a test, but overall, I think it was a fair representation of coverage.
Speed tests were good (mostly).
Below is a chart of my findings, complete with download and upload speeds. I used two different speed tests, and got comparable results from both, so I’m listing the worst speed for each spot check.
Durham Dr, Hoschton
Oak Grove Church, Jackson Trail, Hoschton
Jefferson Memorial Stadium, Jefferson
Real Deals, Jefferson
Brockton Loop (Far Side), Nicholson(?)
Thyatria Brockton Road, Jefferson
Diamond Hill Church Road, Maysville
Sims Bridge, Commerce (Banks County)
As you can see, the Karma Go on the Sprint LTE network provided broadband speeds throughout my journey, but they weren’t always high speeds. The finding near the Real Deals store concerned me, but it may have been related to where I was parked. Overall, it may be worth investigation as a replacement for home internet services within certain contexts (explained below).
Possible Pros for Karma Go ‚Äď Neverstop plan
If you’re a Jackson County resident and looking to escape from the Windstream experience, the Neverstop plan has some benefits that may make it worth considering.
- They offer unlimited internet connection for $50/month, capped at 5Mbps downup (I think it’s 5Mbps both ways).
- The plan allows for three devices to be connected to the Internet at a given time; if a fourth device connects using the plan, one of the other ones will be disabled until the number gets back to 3.
- They offer a 45 day return period for the device, which retails at $149. I don’t think that refund covers any purchased data, but if you feel like the device CAN’T replace your home internet service, will work as a reasonable backup, they let you swap plans at any time.
- They also have a referral program; here’s the link to my code: https://yourkarma.com/invite/stuart4873. If you decide to purchase the device, you get $10 off, and I get a $10 credit.
Possible Cons for Karma Go ‚Äď Neverstop plan
Here’s the possible problems that I see with using Karma Go as a replacement for your home broadband service:
- LTE technology is based on cell service, which is usually less stable than wired connections. Given the instability of Windstream, however, I don’t know if that’s true or not. My single sample today may not accurately reflect coverage at your home; that’s why I’d recommend trying it for 30 days before dropping your old service.
A hotspot is NOT a router. This device allows you to connect a computer or other device over WiFi to the Internet, but not to each other. In other words, your devices won’t see files on each other directly; if you’ve got advanced networking needs or a lot of devices, this may not be a great fit.
- A perk of this is that each device that’s connected gets its own 5Mbps pipeline; that pipe is not shared among your devices.
- You may be able to purchase a router that allows you to create an internal Wifi network for your devices, and then connect over another Wifi channel to the hotspot (like this one). You’d get connectivity between your devices (and override the 3 devices connected to your Karma Go rule), but they’d all split the 5Mbps pipeline.
- The company itself is young. Karma’s only been around for a few years, so there’s no telling how reliable they are. They’ve had some shipping problems and communication problems of their own, but that’s part of being a startup.
Final Thoughts: Should You Buy It?
If you’re unable to get reliable service from your provider, and you have good coverage from Sprint, and you have a limited number of network devices, it’s probably worth your time to investigate the Go as a replacement. Again, a 45 day return period gives you a lot of time to put it through its paces. I love it as a backup option (the Refuel plan), but it sounds like many of you should give it a whirl as a replacement.
I’ve done a lousy job of encapsulating my thoughts about the Professional Association for SQL Server’s Summits for the last few years, but here’s a quick thought about one of the reasons I love this conference: the stretch.
What do I mean by stretch?¬† It’s the exposure to new ideas, new concepts, things I may never use.¬† It’s easy for technical people to get obsessed about our struggles of the day; we invest a lot of mental energy into figuring out problems that require immediate solutions, so we often pick educational sessions that fit into our current solution requirements (i.e. I’m having issues with ETL, so let me go pick up a few sessions on BIML).¬† That’s satisfying, but it’s not a stretch.
The stretch is thinking about issues and problems which I don’t see ahead.¬† It’s the creative detours, the opportunities to explore ideas that are just, well, fun to think about.¬† There’s two benefits to this; first, by allowing the mind to wander, it actually gives me a chance to have an inspirational moment about my current issues (the “aha” moment).¬† Second, by learning a little bit about something foreign to me, it gives me an advantage if the situation ever arises that I may need to know something about that topic (i.e. we don’t use Azure now, but we might in the future).
What’s your stretch at Summit?¬† I’m going to squeeze in a class on R, and maybe some USQL.
Just a quick post as I’m packing up for the Professional Association for SQL Server Summit 2015; this year, I’m carrying a mobile hotspot with me: my Karma Go.¬† Just to state the obvious, yes, I am aware that the Washington State Conference Center has free wifi. I also know that¬†wifi gets horribly overloaded in certain areas (like the keynote rooms) when thousands of device-carrying database people begin tweeting at once. ¬†I also know that most people have hotspots on their phone, but I don’t (corporate phone; unlimited data, no sharing).
So why the Go? ¬†Besides the fact that I want to stay connected with more than just my phone, I also like the fact that it offers a reward for sharing. ¬†You see, Karma’s data plan consists of two parts:
- Pay-as-you-go data. ¬†I filled up with a bunch of data (mostly to use when my home internet goes down), and I refill when I run out. ¬†No monthly subscription, ¬†so I’m not paying twice for unused Internet.
- Sharing earns data. ¬†If a new user connects to my hotspot, they get 100MB of free data, and I get 100MB of data added to my account. ¬†Easy-peasy (my SSID is “Free Karma By @codegumbo”) .
Checking the coverage maps (Karma runs on Sprint LTE), it looks like I’ll have great coverage. ¬† Let’s see if I can stay connected through the keynote this year ūüôā
True story: when I was in college, I was the lead singer (if you can call it that) for a goth rock Christian rave band called Sandi’s Nightmare.¬† I had zero musical talent, and relied solely on the strengths of my band-mate (Brad; you still rock); however, while my singing ability was dubious at best, I was pretty theatrical.¬† We had a total of 3 awesome shows before we moved on to other things, but it was a lot of fun while it lasted.¬† Our biggest hit was a cover of Lust Control’s Dancing Naked, a fantastic little number about David’s celebration of the ark returning; his passion and excitement overwhelmed his embarrassment.
I promise, I’m not making this stuff up; it was pretty edgy for the Bible belt of ¬†Northeast Louisiana in the early 90’s.¬† I mean, I had an earring and everything.¬†¬† The point of this post (besides reveling in my musical abilities and tastes of 25 years ago) is that sometimes we just need to be naked.
Let me explain; I made choices that in hindsight are difficult to defend (like my haircut). However, there was a reason for the choice at the time, and even though it may not have been the greatest idea, I’ve learned something from every experience I had.¬† While there are things that I would do differently if given the choice and knowledge I have today, I don’t regret many things.¬† That’s what the metaphor of nakedness is all about; being open and honest about not only your successes, but your choices along the way.¬† You can’t change the past, but you can learn from it.
Becoming a Naked Developer
I’ve spent most of my professional career as a SQL developer working for one company (13 years in November);¬† I was the first developer hired by that company, and my job was to build an integration platform transforming syslog data into reports, all on a shoestring budget.¬† I strung something together in a couple of weeks, and in the first month, I took a process that normally took two weeks to do by hand down to a day; my boss threatened to kiss me (I was flattered, but declined).
Was it pretty (the code, not the boss)?¬† No.¬† Was it successful?¬† Yes.
Fast forward a couple of years to 2004: I’m a member of small team building a solution to analyze more data than I’ve ever seen in my life (terabytes of data from firewalls and Windows machines); we had to import and semantically normalize data in near-real-time.¬† We built a solution using DTS, and then moved it to .NET and MSMQ.
Was it pretty? No.¬† Was it successful?¬† Yes.¬† Part of that code is still being used today, and my company has thrived off of it.¬† We were acquired by a larger company 8 years ago, and we’ve kept moving forward.
Every few years, we’d talk about doing a rewrite, and we’d bring in some outside expertise to evaluate what we’ve done and help us come up with a migration plan.¬† Some consultant¬†would come in, look at what we did, and shake their heads at us.¬† Each time, we (the remaining original developers) would attend meeting after meeting as our code¬†was dissected and judged; we sat naked (metaphorically) in front of a team of experts who¬†told us¬†everything that was wrong with the choices that we made.¬† At the end of each visit, we’d regroup and complain about the agony of judgement. In all honesty, we would have done things differently, given¬†our current¬†experience (and modern technology).¬† However, the simple truth is: we won the day.¬† We continue to win the day.
During some of these initial code reviews, the consultants were jerks.¬† They’d spend a lot of time bemoaning how horrid everything was; I’d be embarrassed and try to explain what I¬†would have done differently if only I had the time.¬† The consultants¬†would eventually propose some complex solution that would cost hundreds of thousands of dollars to implement.¬† Our company would consider the offer, stick it in a file somewhere, and we’d keep on keeping on.¬†¬† Clunky code that works is better than expensive code with the same outcome.
Over time, I learned to live with the embarrassment (as you can tell from this post); I started listening to what the consultants would say and could quickly figure out which ones had good advice, and which ones were there to just be seagulls.¬†¬†People that focused on solving the immediate problems were much easier to talk to than the people who wanted to rip out everything and start over.¬† Don’t get me wrong; sometimes you need to start over, but there is¬† a cost associated with that decision.¬† It’s usually much easier to start where you are, rather than trying to reinvent the wheel.
What’s my point?
First and foremost, don’t dwell on your past mistakes.¬† If you can honestly say that you built something that worked within the constraints of the time period, no matter how clunky or silly it seems now, then it was the right decision.¬† Working software works.¬† Laugh at it, learn from it, and let it go; repair what you can, when you can, but keep moving forward.
Second, learn to code naked.¬† Whatever it is that you’re trying to do at the moment, share it.¬† Open yourself up to code review, to criticism; exposure brings experience.¬† Granted, you’ll meet some seagulls along the way, but eventually, you’ll figure out who’s there to help you grow as a developer.¬† Foster those relationships.
Third, encourage¬†people to¬†grow; recognize that¬†the same grace you desire for your coding mistakes is applicable to others as well.¬† Be kind, be gentle, but be honest;¬†seek to understand why they made the design choices they did, and be sensitive to the fact that if it works, it works.¬†¬†Sometimes we get caught up on trying to figure out¬†the right way to do something, and ultimately end up doing nothing because it’s too¬†difficult to implement it according to the¬†textbook.
Life is too short to not enjoy what you do; try to share that joy without embarrassment.
October 8, 2015
¬∑ stuart ¬∑ 3 Comments
Tags: community, Development, personal, Professional Development, SQL Server, Troubleshooting ¬∑ Posted in: Code, Good Habits, Professional Development, SQLServerPedia Syndication
Got into a deep discussion with a colleague tonight over different approaches to Agile development (yes, I‚Äôm a geek; why do you ask?).¬† I‚Äôm a Fixed Length Iteration kind of guy, particularly since I‚Äôve spent most of time on the Operational side of the house recently.¬† He‚Äôs a Variable Length Iteration fan, and I thought the arguments on both sides of the fence were compelling enough that I wanted to blog about them.
Really, I‚Äôm doing this for my adoring fans.
Actually, I‚Äôm doing this because I feel guilty about not blogging, and thought this was at least SOMETHING to warrant paying for a domain year after year.
Anyway, here‚Äôs the breakdown of the two arguments; let me start by spelling out some common assumptions.¬† I‚Äôm assuming that you‚Äôre reading this because you have some interest in development, and more specifically, some interest in agile development.¬† If you have those interests, then I‚Äôm also assuming that you know what an iteration is.¬† If not, the hyperlinks in the previous statements should help educate you and steer you clear away from this blog to more fascinating subjects.
The Argument for a Fixed Length Iteration
There‚Äôs a couple different versions of the Fixed Length Iteration, based on either a calendar (January, February, etc.) or a cycle (every four weeks); the goal is to commit to a ship date, and stick to it time after time.¬† I‚Äôm more of a fan of a calendar-based Fixed Length Iteration; the actual length of the iteration varies (February is short), but development is consistently wrapping up every month, and every month code is being shipped out the door.¬† My reasons for supporting this model are as follows:
- People are cyclical creatures. We live from paycheck to paycheck; we hate Mondays and work for the weekend.¬† Having a fixed length cycle (with some minor variation if you based that cycle on a calendar) helps in estimating and planning what gets done and when.
- A fixed cycle forces developers to think in terms of components. If you‚Äôre expected to ship working code at the end of each month, you start writing code that is bite-size, so that you can leave out pieces that aren‚Äôt finished (rather than waiting until the whole thing is done).¬† Bite-sized code is easier to test and deploy with reduced risk.
- A fixed cycle means that it‚Äôs easy to change directions without stopping development efforts. Sometimes business priorities change; a fixed length cycle allows developers to continue moving forward until the end of the iteration and change gears there.¬† The start of a new iteration is never far away, so most businesses can wait until the next month rather than waiting till an unknown end of a sprint.
The Argument for a Variable Length Iteration
I‚Äôm trying to give this perspective a fair shake, even though I don‚Äôt subscribe to it; in a variable length iteration model allows business and development to scope out work at the beginning of an iteration and estimate a ship date regardless of the cycle or the calendar; the goal is to allow code to mature and be more stable.¬† My friend subscribes to this model because:
- Variable length iterations resemble traditional projects. There‚Äôs a scope estimate, requirements gathering, and work estimates (and traditional slippage).¬† Most agile purists immediately scream ‚ÄúWATERFALL‚ÄĚ, and there‚Äôs some truth to that, but a variable length iteration is comfortable to business.¬† It‚Äôs like Mr. Roger‚Äôs house shoes.
- They lend themselves to continual shipment of code. If the iterations are variable, developers can focus on one task from start to end, and begin to operate on separate iterative cycles; if you have a development staff of 5, you could theoretically have up to 5 separate iterations going on at the same time; when a developer finishes, they ship their contribution without waiting on the sprint to end.
- This fragmentation of the iteration allows for sudden, non-disruptive change. If there are multiple iterations occurring for each independent block of code, and business needs change for one line, then only that line has to shift gears.¬† There‚Äôs no impetus to wait; you stop where you are on the one piece, and move on to the next piece.
I‚Äôd love feedback on this; as I‚Äôve stated from the outset, I‚Äôm a fixed length iteration guy.¬† What did I miss?¬† Are there other benefits to the Variable Length model that I‚Äôm missing?
Been really freaking busy the last few months, so I haven’t had the chance to share the news of a few upcoming presentations:
On October 10, I’ll be presenting at the Orlando SQLSaturday, doing my session on a DBA’s guide to Hadoop.¬† I’ll be representing that session at the¬†Professional Association for SQL Server¬†Summit in Seattle (October 27-30).
Brief service announcement is over; now I can say I’ve blogged recently.