What the Business Wants: FEATURES (TSQL Tuesday #13)

It’s TSQL Tuesday Again…

This month’s #tsql2sDay is hosted by Steve Jones (blog | twitter), and the topic is “What the Business Says is Not What the Business Wants.” Steve asks the question:

What issues have you had in interacting with the business to get your job done?

I thought about this for a long time. Eventually I realized that I wanted to see the pattern in the trees from the last ten years.

What the Business Says: “We Want Features!”

As a DBA working with software developers, I’ve heard many times from the business that they want new features.

They need a new feature to land a big client, to be the first to market with something, to retain a client, to compete with x.

They need a feature to land on Mars, and if we can’t do that reliably, well, we should just do that once. They don’t care what it takes to get to that feature, we needed to get it last week.

And so the wheels squeak, and the features are prioritized with a mysterious calculus involving the business which is never fully revealed.

But although I have seen that the business does not always dictate the priorities directly and fully, I have  often seen that the business dictates the timeline for what is prioritized. And that timeline is usually pretty quick.

What the Business Wants: They Actually Do Want Features. They Just Don’t Want to Talk about Sacrificing Scalability/Availability/Recoverability for Speed of Delivery.

The thing is, the business is telling the truth. They do need features to make money– at least in the competitive markets I’ve worked in.

The Scalability Fairy sadly doesn't show up nightly when your application loses a tooth. Or a customer.

However, typically people are so busy stressing when they need the new features that no honest assessment and agreement on quality is reached.

Maybe people think it’s embarrassing to have a frank discussion about what type of SLA and reliability will come with a new feature. Possibly that opens a company to some legal challenges if a different level of service has been provided to the customer. However, this is exactly what the business should truly want: to promise the correct level of reliability to the customer.

Now, not all new features need to be high quality work. Sometimes you’re delivering a shiny toy, and nobody’s going to use it to shave, wear it as a diaper, or drive it to work. But a feature needs to have a clearly defined level of scalability and it needs to be safety-proofed to the appropriate level for that product. It shouldn’t enrage your large customers on a regular basis.

Commonly, this part of the process is overlooked. People don’t want to talk about it. Unfortunately, it leads to delivering a product that  may look good at first on the outside, but may not really please the customer until it’s been patched up in the middle of the night a few times, retrofitted on a weekend, and ultimately largely re-written and re-released.

So, What Do you Do?

As an operations person, you bring up the conversation about SLAs repeatedly and make sure all parties come to an agreement. You ask questions about disaster recovery and availability and make sure you have the time and resources to meet those needs.

You make sure there’s time in the release schedule for performance and load testing, and  you set your expectations to medium-low until you have strong data that shows how something will perform. Trickiest of all: you somehow get that data prior to release.

You document everything, because people won’t remember later.

But most of all, you establish a good relationship with people in the business by being genuine, open, and honest with them, and not pre-judging them. If your business people trust you, they are likely to ask you a few questions before making many promises.

Introduction to Partitioning: Resources!

This past Saturday I presented my shiny new Introduction to SQL Server Partitioning session at SQL Saturday 61.

There were two separate sessions scheduled for partitioning that day, but the room was still full– thanks everyone for coming to learn about how, why, and when you might use this feature.

Slides, Links and Scripts

My slide deck, Introduction to SQL Server Partitioning, is available for download.

Links to all sorts of resources about partitioning and my simple partitioning demo script are available here:
http://littlekendra.com/resources/partition/

Whose Doll is That?

My partitioning slide deck features a lovely assistant who is sawed into partitions, and also offers pieces of advice.

This little beauty was found on Flickr and is the fine work of Lara604.

Cheerfully Demonstrating Partitioning of the Head

Review: A day of doing many things at once with @AdamMachanic

A day of doing many things

At SQLPass this year I was fortunate to attend “A day of doing many things at once: Multitasking, Parallelism, and Process distribution” by Adam Machanic (blog | twitter). This was a day long post-conference.

So, how was it?

This was a fantastic seminar. There was a really good flow to the talk, which started in CPU background and architecture, then moved through Windows Internals, SQL Server internals, and on to specifics of parallelism in queries. Then we finally moved on to administration topics, as well as different methods of process distribution. A full outline of the day is here.

I think the presentation worked very well because of the balance of theory and practice. Essentially, there was a very good ratio between ‘what, ‘why’, and ‘how’.

I’ll look back at the outline for this seminar when designing longer presentations myself.

Did I learn anything useful?

Yes! The information on plan shapes and tricks to manipulate them was incredibly interesting, and is something I know will be useful. I also learned some interesting specifics about how the DAC works, and have a much more holistic view of how SQL Server uses processors and parallelism. Check out my tweets below for a little more insight into what my day was like!

Free webcasts. Yep, free.

Adam has some webcasts on parallelism available for download which you can watch for free.

My tweetstream from the session…

Here’s what my day was like, according to Twitter.

• Postcon fun with @AdamMachanic today! #sqlpass Processes do not run, *threads* do.
• Quick discussion of fiber mode for SQL Server: very limiting (http://bit.ly/bn6RoK)
• Thread starvation: pre-emption by high priority threads can prevent some threads from ever running.
• Threads running on client OS get a smaller amount of quantum units than on a server os (more frequent interrupt frequency)
• Three types of processor affinity: none, hard affinity, and ideal affinity
• Lots of love for sysinternals (http://bit.ly/WPxha) and theCPU-Z tool (w/ props to @BrentO for recommending http://bit.ly/1iBcg6)
• Interrupt counts include not just when a quantum expires, but also when a thread finishes.
• Lots of cool WMI queries being run from inside SSMS
• Mine is still getting even better RT @whimsql: Amen Tom! RT @SQLRockstar Best. Summit. Ever.
• Meeting the SQLOS! It’s a “cooperative” scheduling system: everyone’s equal
• SQLOS provides an abstraction layer so storage engine, qp, etc can all talk to it instead of directly to the OS
• Proc Affinity at sql server level may be worth testing w/ multi instances. WIth virtualization taking predominance, is less common.
• Differences between resource waits and signal waits being explained
• 484 Wait types in SQL Server 2008– plug for #sqlhelp hash tag for those with limited documentation.
• I totally just got called on in a “what feature uses a hidden scheduler” Pop Quiz. #FAIL
• @PaulRandal yep, we were all “so THAAAAAAAT’S how that works.”
• Don’t think of operators in QPs as being parallelized. Think more of each set of rows as being prallelized.
• Very few iterators are actually parallel-aware. Most do not need to be, even if being used by parallel streams.
• OH: “I trust myself, but I don’t know if you should.” <– always an appropriate comment when referring to production environment
• And now we return to our discussion of the “Big O” and the Query Processor.
• We just covered tempdb spills and @crysmanson ‘s old enemy, the resource_semaphore wait type.
• Few outer rows demo showing repartitioning scheme and rows redistributed on threads– very cool
• Verrrrrrry interesting stuff with CROSS APPLY and parallelism
• Cost threshold for parallelism default is still what it was set originally in 7.5, for many contemporary systems it may be too low.
• And that makes me happy to hear since we do raise the default cost threshold for parallelism on our prod servers
• @AdamMachanic just actually turned it up to 11.
• If you hit THREADPOOL waits, don’t just up the max worker threads permanently, find the root cause for the situation.
• Finishing up with a monitoring parallelism section — really nice flow to the talk today!
• Piles o’ DMV fun, including the reason sys.dm_exec_requests has some funkiness: it shows wait state only for the root task
• @AdamMachanic is demoing how sp_whoisactive will display your wait types, find your tempdb contention, and wash your dishes.
• Demo of manipulating memory grants to cause a query to spill to tempdb purposefully… we’re not in kansas anymore.
• @TheSQLGuru I’ve enjoyed it a ton– great combo of really interesting demos and information.

The Next 24 Hours of PASS Event: Announcement and Call for Presenters

Tonight I checked in on the Twitterverse and saw that Jason Strate announced a new blog post about the next 24 Hours of Pass event.

This time #24HOP is looking for presenters! Check this out:

…the next 24 Hours of PASS will feature 24 women speakers speaking on SQL Server. The 24 women speakers haven’t been selected yet. In fact, if you happen to have and abstract in mind or know of a speaker that should speak, send an e-mail over to24hours@sqlpass.org. The deadline for abstracts submissions (max 250 words with a 125 word bio) is January 14th.

The event will occur during March, which is Women’s History Month.

In the past I haven’t seen a call for speakers for #24HOP, so this is quite a change.

Suggest Someone, or Yourself!

I know some women I’ll be suggesting for the event. Will yours be the same, or different?

Read Jason’s full post here.

#SQLSat61, DC Edition: Free SQL Server Training in Reston, VA on December 4

I’m Going to See the President

OK, this isn’t really true. Unless the President comes to SQLSaturday. It’s free, and he’s invited like everyone else.

I’m Talking About Partitioning!

This part is all true. And it’s awesome.

I’m giving an introductory session on horizontal table partitioning in SQL Server, and I’ll dish up all the goods you need to know about the feature.

Here’s the deal with table partitioning: it can be extremely beneficial to businesses in the right situations, and when specific changes to schema and application queries are possible. But if you’re doing it wrong, things can get ugly pretty fast. How do you know where it’s a good choice and what you need to do? It’s complicated.

In my talk I’ll cover all the basics of the feature, we’ll talk about requirements, pros and cons, gotchas, and the implications of partitioning for applications. I’ll give you what you need to be a scalability hero.

Is It Hard to Get To?

Not at all, my friend. This event is in Reston, VA and is very close to Dulles airport. For other out of towners, there’s lots o’ great flight deals into Dulles. And there’s a great slate of speakers signed up, so come on out.

How Do I Sign Up?

Click here for more information on the event.

Hope to see you on December 4.

SQLPASS Day 1 and 2: Please Focus Your Attention on the Command Prompt

Words to Live By

A sign will be henceforth posted on my office door which reads: “PLEASE FOCUS YOUR ATTENTION ON THE COMMAND PROMPT.”

ooooo.... ahhhh...

And I will always make sure to remember to be careful when working with the Engine of the Devil.

Get Your Conference On: They’re Selling DVDs

If you’re a first timer at SQLPass, buy the DVDs. They’re $125 during the summit, and they free you up to take time to meet others at the conference, use the Ask the Experts and SQL CAT resources, and generally do all sorts of things you can’t do at home later.

Take Time to Ask the Experts (4th Floor and Developer Pods)

Developer Pods are on the sixth floor with bright yellow chairs– people from the “Ask the Experts” area have been deployed to PODS throughout the convention center.

I had a great visit to the Ask the Experts room last year and got some great ideas talking about problems and questions in my production environment, and highly recommend meeting the experts before you leave.

We’re Going Parallel: Project Madison Grows up

In Ted Kummert’s keynote on Day 1, he announced that SQL 2008 R2 Parallel Data Warehouse has now RTM’d and appliances are available for purchase. The appliances were on stage and Jesse Fountain demonstrated creating a table, connecting to PDW, and loading data.

Demo: based on a 100TB database, created and loaded in two days on a four rack system. Query that runs against 800 billion rows of data. Dynamic process monitor shows processor usage of all the cores, and you can view the health of the physical nodes. This was all in what looked like a Reporting Services interface.

Takeaway: DBAs love hardware, and this had us all drooling. A netbook was given away to to a lucky attendee with a ticket under their seat. Next year, can we win a Madison appliance?

Overall, it’s hard to say from a short demo how this appliance would fit with your workload. But it’s easy to want to learn more.

The Cloud: Still Here! Even Cloudier!

We’re used to talking about the cloud each year, and this year is no different.  Today Ted talked about the Azure platform – emphasis was on providing “a consistent platform, from server to service.”

A new community technology preview is coming with Web Administration, Reporting, and DataSync.  We saw demos of SQL Azure reporting.

Takeaway: Cloud services are maturing and moving along. This continues to look like a great solution for some businesses, and as tools are added for manageability and reporting, this will become adopted by more people.

DENALI: CTP Available!

First off, the CTP for Denali is available here: http://www.microsoft.com/sqlserver/en/us/product-info/future-editions.aspx

Denali features that I’m most interested in include:

• High availability and disaster recovery features grounded in Windows Clustering. This is the ‘AlwaysOn’ set of high availability features:
  • Tools in Management Studio
  • Wizards allow you to create a “management group” for specific databases. This group will be failed over as a unit.
  • You can fail over to multiple secondaries (up to four) using mirroring.
  • Shared disk can be used.
  • Some secondaries can be set to manual failover/asynchronous, while others are set to automatic failover/synchronous.
  • Secondaries can be readable.
  • Your backups can be offloaded to secondaries!
  • Wait, do you need any more? You don’t, do you? DROOL.
• Provide new a unified developer experience in Visual Studio, and also in a standalone tool. All BIDS projects will be unified together with a new database project type in a single development environment.
• Columnar Indexes: faster reads, improved query processing.
• Filetable:
  • Files can be imported into SQL Server
  • The files can still be previewed in an interface on the Windows system, although they reside in SQL Server
  • The files can be indexed and managed in SQL Server, attributes are maintained in SQL Server
• PowerPivot engine moves into Analysis Services
• Developer tools: new toolkit in Visual Studio, Project Juneau, for development.