All the headlines and interesting SQL Server information that we've collected over the past week, and sometimes even a few repeats if we think they fit. These headlines are gathered throughout the week and are posted in real time at the website. Check there for information throughout the week or enjoy this weekly summary of the SQL Server world. WebinarsThe 2019 State of Database DevOps results, live with Donovan Brown! - Donovan Brown, Principal DevOps Manager at Microsoft, joins Steve Jones, Microsoft Data Platform MVP to discuss the latest in all things Database DevOps. They offer a closer look at the key findings in the 2019 State of Database DevOps Report, and investigate the growing importance of the database in successful DevOps and IT performance....(more) Why the database is at the heart of DevOps success - DORA’s recognition of the critical importance of the database to DevOps in their Accelerate State of DevOps Report should act as a timely wake-up call to those companies that still see DBAs and developers as operating in separate silos. Join Microsoft Data Platform MVP and SQL Server Central Editor Steve Jones to discover how you can build a common understanding and atmosphere of collaboration....(more) DevOps: What, who, why and how? - More and more organizations are turning to DevOps as a way of working together to improve the efficiency and quality of software delivery and start adding more value to the business. But what exactly is DevOps and what does it mean for you and your organization?...(more) Virtualization and ContainersKubernetes Fundamentals: Facilitating Cloud Deployment and Container Simplicity - Kubernetes (sometimes abbreviated to “kube”) is open-sourced, was originally developed by Google, and organizes containers into logical units for transport and use in the cloud. Containers support the construction of self-contained environments capable ......(more) Deploying SQL Server to Kubernetes using Helm - In previous posts I’ve run through how to deploy sql server to Kubernetes using yaml files. That’s a great way to deploy but is there possibly an easier way? Enter Helm. A package manager for Kubernetes. Helm packages are called charts and wouldn’t you ......(more) Vendors/3rd Party ProductsNew Case Study: Reducing capital by $500,000 with SQL Clone - When a well-known financial services company was searching for a solution to move data through the development cycle faster and easier, SQL Clone provided impressive results. As well as leading to improved and more in-depth testing processes, SQL Clone saved between $400,000 and $500,000 in one month alone by reducing capital expenditure. Read more about their journey in this new SQL Clone case study. ...(more) Compliant Database DevOps for SQL Server using Redgate Tools - To many people, Database DevOps, means automating the processes involved in the provisioning, development and release of databases. This focus on automation is important, of course, because it allows us to deliver releases to our users reliably and frequently. In this article Stephanie Herr walks through the Redgate tools and how they can help your technology teams work more collaboratively towards a shared goal....(more) T-SQLEssentials and usage of NOLOCK hint in SQL Server - The main idea of the SQL Server locking mechanism is that it controls the consistency of transactions. According to this principle, if a process wants to perform insert, delete or update operations, SQL Server engine locks the row or rows and does not ......(more) Common data-type conversions between SQL Server, Oracle, Sybase ASE, and DB2. - SQL Server includes a little-known, but handy, function that can show you the equivalent data type in a target system; useful for ETL between disparate systems. Run this code: DECLARE @source_dbms sysname = N'%' ,... ...(more) Getting Parameters Out From a Stored Procedure–#SQLNewBlogger - Another post for me that is simple and hopefully serves as an example for people trying to get blogging as #SQLNewBloggers. One of the lesser used and known features of T-SQL are the output parameters from a stored procedure. I used one of these recently, ......(more) Failure is always an option - A while back, I had the pleasure of sitting down to talk with Bert Wagner (blog|twitter), Erin Stellato (blog|twitter), and an anonymous friend about some of our #SQLFail moments. Everyone screws up Everyone has their failures. Nobody is perfect. But ......(more) ToolsSQL SERVER – Export Data From SSMS Query to Excel - Today’s blog post is actually a follow up of my earlier blog post SQL SERVER – Copy Data From SSMS Query Result to Excel. In the earlier blog post, we had discussed how to copy data from the resultset. In this blog post, we will discuss how we can directly ......(more) A brief introduction for bcp - I end up using the bcp utility to move tables between environments, and I’ve found that a lot of the guides on the internet go into great detail on how to use it. That’s fine if you have a lot of time. Here’s my brief introduction so that if I forget ......(more) Multi-server scripts with SSMS using SQLCMD mode - SSMS is the most common tool used by SQL Developers or DBA’s but unfortunately there are some features that are barely known, SQLCMD mode is not the exception and that’s why I decided to follow up in this topic. You may want to check one of this previous ......(more) The Lighter SideOff Topic: Charity and snooker - So this will be a very unusual post for this blog. It will be completely off topic. One of the things I do in my spare time is to play snooker. Mind you, I am not good at the game. But I enjoy it, and it’s a nice break from sitting at my laptop. Marathon... The ......(more) Relativity Space to launch from historic Florida site - A company that aspires to 3D print almost the entirety of its rockets has reached an agreement with the US Air Force to launch from historic facilities ......(more) SQL Server SecurityQuick Permissions Audit - Whether it is for a client, an audit, or just for good housekeeping, DBAs will often need to figure out who has access to what. In addition, they may need to know by what means people have access within your databases. When that need arises, it is frequently ......(more) SQL Server on LinuxPro SQL Server on Linux- More Installs - As part of my learning goals for 2018, I wanted to work through various books. This is part of my series on Pro SQL Server on Linux from Bob Ward. After my last post, I decided to try and install the SQL Server 2019 preview version and see how that worked. ...(more) Software DevelopmentMeasuring Technical Debt - Recently I saw a Twitter thread bemoaning the rise of ‘psychological safety’ as a phrase. Not because it isn’t important – it is – but because giving it a name has made it acceptable and risks giving a scapegoat to hide behind. There’s the possibility ......(more) Microsoft Lowers GitHub Private Repositories Price to Free - GitHub makes private code repositories free, while combining its cloud and server products into a single offering aimed at hybrid cloud. ...(more) Security news and thoughtsMonster 773 million-record breach list contains plaintext passwords - Have I Been Pwned, the breach notification service that serves as a bellwether for the security of login credentials, has just gotten its hands on its biggest data haul ever—a list that includes almost 773 million unique ......(more) The Security Challenges of Moving ERP to the Cloud - A new report from the Cloud Security Alliance examines how organizations are moving business-critical ERP applications to the cloud and potentially exposing themselves to new risks. ...(more) Linux's Systemd Hit With Three Security Holes - Red Hat has issued patches for the two most serious security vulnerabilities discovered in Linux's systemd. ...(more) R LanguageSolving Naive Bayes With R - This is part four in a series on classification with Naive Bayes. Classification Of Features With R So far, we’ve walked through the Naive Bayes class of algorithms by hand, learning the details of how it works. Now that we have a good understanding ......(more) Automated Dashboard for Credit Modelling with Decision trees and Random forests in R - CategoriesProgrammingTagsData VisualisationFlexdashboardRMarkdownIn this article, you learn how to make Automated Dashboard for Credit Modelling with Decision trees and Random forests in R. First you need to install the `rmarkdown` package into your ......(more) Product Reviews and ArticlesHow to Monitor IDENTITY columns to prevent unplanned downtime - We all try hard to plan for data growth and changing workloads, as we build and adapt our SQL Server systems, and to anticipate potential problems. Some issues, however, take a long time to appear. They lay in wait, unnoticed but not dormant, and if ......(more) Automating Builds from Source Control for the WideWorldImporters Database - Sample databases from Microsoft are like YouTube stars: they’re quirky, popular with their fans, and always online somewhere. It makes them a common choice when it comes to creating prototypes, but their lovable quirkiness sometimes makes this tricky. ...(more) PowerShellCloning Databases with PSDatabaseClone - Guest Author : Sander Stad (@sqlstad) PSDatabaseClone is a PowerShell module that has the ability to create images of databases ("clones") and distribute those clones to one or more hosts. Why use this module? Database administrators, and really anyone ......(more) PowerPivot/PowerQuery/PowerBIBuild a Slicer Panel in Power BI - Some folks had questions about how we built the slicer panel in Power BI from a previous video. In this video, Adam walks you through how to do exactly that in Power BI Desktop. The post Build a Slicer Panel in Power BI appeared first on Guy in a Cube. ...(more) Time Animation with ArcGIS Maps in Power BI - (2019-Jan-14) Dynamic maps with dynamic visualization usually attract my attention, especially when I look at the satellite images of various cloud formations in the upper levels of the atmosphere. Heatmaps could be another example where animation could ......(more) Link from a Power BI Paginated report to a Power BI report - Patrick looks at how you can add a link to your Power BI Paginated report and open a Power BI report while passing filters over to explore data. The post Link from a Power BI Paginated report to a Power BI report appeared first on Guy in a Cube. ...(more) A further (final?) update from Seattle - This is the longest running blog post series I have on my website. Why does Power Pivot Crash so often and what can I do about it? I have three previous articles that you can read in the links below Original post (Oct 16): Does the Power Pivot Excel ......(more) Power BI Data Privacy Levels And Cloud /Web-Based Data Sources Or Dataflows - I have already blogged in great detail many times about Power BI/Power Query data privacy settings (see this series for example) but there’s always something new to learn. Recently I was asked a question by Ian Eckert about how Power BI handles data ......(more) Performance Tuning SQL ServerQuery memory grants part 3: How much memory can one query get? - When you build a brand new shiny SQL Server, you want to get a lot of memory so your queries run faster, right? More memory for your data pages in the buffer pool and more memory for all the queries that keep sorting data in the database instead of the ......(more) Troubleshoot SLEEP_TASK Wait Type - Jobs on a SQL server can take milliseconds or minutes to complete depending on the size and complexity of your query and also the SQL environment on which it’s executing. It’s important to have a baseline or at least be familiar with the normal amount ......(more) When the query plan hash is deceptive - When you’re querying the plan cache, you need to know the four primary objects: the query hash, query plan hash, sql handle, and plan handle. However, the query plan hash recently surprised me. Take a look at this index (in StackOverflow2010) CREATE ......(more) Video: Getting Started with SQL Server Execution Plans - I put this video together in about ten minutes or script prep, and one-take of the video. It is not meant to be an all-inclusive lesson…merely an introduction to how to read SQL Server Execution Plans (Right to Left, Top to Bottom, etc.) and some basic ......(more) Query Store: Showing the Impact of Indexes on Inserts - Introduction It is common knowledge in database circles that indexes improve query performance either by satisfying the required result set entirely (Covering Indexes) or acting as lookups which easily direct the Query Engine to the exact location of ......(more) SQL Server Perfmon counters that are still interesting in 2019 - “Page Life Expectancy? That belongs in a museum!“ (Cross-posted from the PASS Blog.) Years ago, my first go-to tool for performance tuning was Performance Monitor: it was available everywhere, worked in all versions, and gave me a lot of useful information ......(more) Execution plans: RetrievedFromCache - I first came across the value for RetrievedFromCache when I was reading a confusing execution plan. At first, I thought this could be really useful. If this value was true, you could assume that another query had compiled the execution plan that your ......(more) It’s always parameter sniffing (part 1?) - This is a blog post about a perennial issue. Have you ever had a call in the middle of the night about a slow webpage and you know nothing in the webpage changed? Yeah, it’s probably parameter sniffing. First, some clarification All parameter sniffing ......(more) Let’s talk SQL Server Waits: Topic 2 – CXPacket - With our effort to talk about SQL Waits we come across another common wait called CXPacket. Last week we reviewed SOS_Scheduler_Yield. If you see CXPacket waits on your supported system you can safely assume you have execution plans running parallel. ...(more) A real parameterization problem with a plus - A few weeks ago I faced this problem: One query on my application was (fortunately in the development environment) was facing a very bad execution time. Since the query was generated by entity framework, I used SQL Profiler to capture the query with ......(more) Microsoft NewsMark Your Calendars: The End of Support for Windows 7 is Jan. 14, 2020 - One of Microsoft’s most popular operating systems is one year away from dropping out of its extended support lifecycle. This has consequences for customers who depend upon these systems for handling data and keeping devices secure. However, there are ......(more) Windows 7 enters its final year of free support - Enlarge / Licensing and support lifecycles are not really the easiest topics to illustrate. (credit: Peter Bright) Windows 7's five years of extended support will expire on January 14, 2020—exactly one year from today. After this date, security fixes ......(more) MDX/DAXNaming variables in DAX - The current version of DAX has a limitation for names you can provide to variables in a DAX expression: a variable name cannot be the name of a table in the data model. This is not a big issue if you create DAX expressions after you imported all the ......(more) Using the DAX Calculate and Values Functions - The series so far: Creating Calculated Columns Using DAX Creating Measures Using DAX Using the DAX Calculate and Values Functions If you should ever start reading a book on DAX, you will quickly reach a chapter on the CALCULATE function. The book will ......(more) Hardware NewsQuick Look: Using WD's Black SN750 SSD as a Thunderbolt 3 External Drive - External SSDs with a Thunderbolt 3 interface are still the only option for users looking to get high-end performance in a bus-powered portable device. These SSDs have become possible due to the emergence of NVMe drives in the compact M.2 form-factor, ......(more) Toshiba Forgoes Next-Gen Recording Tech in Designing Its First 16TB Drive - Microwave and laser-heated hard drives may be the future for higher capacity, but Toshiba promises denser scale-out and cloud storage achieved with incremental technology tweaks. ...(more) Using Intel Optane Storage for SQL Server - The Intel Optane SSD DC P4800X Series was introduced in Q1 of 2017, initially with a 375GB capacity and later with a 750GB capacity being released later in 2017. Intel has also recently released a 1.5TB version of ......(more) More CPU Competition Coming for Intel - On January 9, 2019, AMD CEO and President Dr. Lisa Su presented a CES 2019 Keynote where she demonstrated (at 1:25:00 in the video) a 7nm, eight core/sixteen thread, 3rd Generation AMD Ryzen “Matisse” desktop processor running the Cinebench R15 Multithreaded ......(more) HA/DR/Always On/ClusteringAvailability Groups: When Your Synchronous Secondary isn’t Synchronous - During a recent DR scenario I came across a slight misinterpretation in how availability groups with replicas in synchronous commit behave. How An Availability Group With A Synchronous Secondary Should Behave Under Normal Circumstances Normally, if we’ve ......(more) ETL/SSIS/Azure Data FactoryCSV Row Counts - This is one of those posts where if my opening premise is wrong, the whole thing is a moot point. I've been wrong before, so let's roll! There doesn't seem to be an efficient way to get a count of the number of records in a CSV file. Every record can ......(more) Grant SSIS Catalog Read-Only Permissions - I like the SSIS Catalog. I think it’s a snappy SSIS Framework. I write this as someone who has been developing data integration frameworks since the days of DTS. The SSIS Catalog manages permissions internally. This fact can cause stress for DBAs used ......(more) DevOps and Continuous Delivery (CI/CD)DevOps and Culture, part 1 - Premier Developer Consultant Ron Vincent explores the importance of an organization’s culture in DevOps (part 1). Peter Drucker famously said, “Culture eats strategy for breakfast.” This is a great introduction to how we think about DevOps and in this ......(more) The ten habits for highly successful compliant database DevOps - Database DevOps has come of age. Now seen as a key technical practice which can contribute to the successful implementation of DevOps, it stops the database being a bottleneck and makes releases faster and easier. Conversely, perhaps, the automation ......(more) The 4 capabilities you need to introduce true database DevOps - The latest Accelerate State of DevOps Report from DORA concludes that successful software delivery unlocks competitive advantages, including “increased profitability, productivity, market share, customer satisfaction and the ability to achieve organizational and mission goals”. The result of over five years of research with over 30,000 data points, it shows a direct correlation between DevOps and better business performance....(more) DevOps and Automation Will Eliminate the DBA - I’ve been reading about the death of the DBA ever since I first made the jump from full time developer to full time data professional. The first time I heard it was when SQL Server 7.0 was released. Did you know that SQL Server 7.0 was self-tuning? In ......(more) Data WarehousingData Warehousing Tip #5 – Use surrogate keys - My fifth data warehousing tip is to always use surrogate keys for your dimension and fact tables. Surrogate keys are keys that are generated internally by the data warehouse. They have nothing to do with the business keys. The business keys are the ......(more) Data Warehousing Tip #4 – Trust your instinct - My fourth data warehousing tip is to trust your instinct when it comes to building solutions, and if to you the solution seems too complex then maybe it is. Sometimes things are just complex. It’s the way that it is. However, sometimes things are more ......(more) Data ScienceClassification With Naive Bayes: An Introduction - This is part one in a series on classification with Naive Bayes. What Is Naive Bayes? Let me fix that right away: Naive Bayes isn’t a thing; it’s a class of things. You do realize that collective nouns are typically referred to in the singular, right? You ......(more) Data Privacy, Complianace, and GDPRApple's Tim Cook Calls for Stronger US Privacy Laws - Apple CEO Tim Cook is arguing in favor of stronger U.S. privacy laws. ...(more) Data Masking is not enough to protect personal information from prying eyes! - Data Masking is the process of either obfuscating or replacing personally identifying information with meaningless data that cannot be used to identify the items being masked. The data masking process is frequently used to alter... ...(more) Conferences, Classes, and EventsUpcoming Free Training Webcasts - Hey, wanna learn about SQL Server for free? Feb 7 – PASS Data Expert Series: PASS is taking the top-rated Summit 2018 recordings and replaying them live, for free, on a webcast. I’ll be in the chat room too, taking your questions. Register here, and ......(more) Webinar: Introduction to SQL Server 2019 - Modern enterprises are struggling to gain insights from an exploding number of database management systems and ever-growing data volumes. SQL Server 2019 can help you overcome the challenges of integrating data and bring AI and machine learning to all ......(more) R Training In Redmond - I had to wait until this was official, but the long wait is over: I am giving my full-day R training in Redmond, Washington on Friday, February 8th. Tickets are priced at $125 for a jam-packed day full of learning and excitement and a little bit of snark ......(more) SQL Bits 2019: The Great Data Heist - SQL Bits is almost here! It’s February 27th to March 2nd. SQL Bits is one of the most popular SQL conferences out there and you still have time to make it! (Although not much, that’s only about a month and a half). As much as I love PASS and PASS Summit, ......(more) Workload Tuning Training - I recently booked my flight to the U.K. for SQLBits, and I realized I probably should write a post about my full day training session on workload tuning! It’s on Wednesday, February 27, 2019, and you can read the abstract here: Stop Tuning Your Queries ......(more) Full-Day R Training In Cleveland, February 1st - SQL Saturday Cleveland is a great event. Each year, that crew puts on a show good enough to convince me to brave the northern Ohio winter, and this year will be bigger than ever with a brand new facility. If you are looking to attend SQL Saturday Cleveland ......(more) Azure SQL DatabaseLesson Learned #59: I would like to upload my own certificate to connect to Azure SQL Database. - Hello Team, Today, I worked on a service request that our customer asked about to use their own certificate to connect to Azure SQL Database. Nowadays, there is not possible to use your own certificates to connect to your Azure SQL Database. Azure SQL ......(more) Controlling the firewall for an Azure SQL DB via T-SQL - The other day I took a Microsoft Learn course about securing Azure SQL DB. It was really enjoyable and I learned quite a bit, but one of the things that really stood out was the fact that you can control the firewall via T-SQL. Now the reason this stands ......(more) Administration of SQL ServerWhat Metrics Does Rebuilding Indexes Improve? - Gall Of A Red Ox For Row Store indexes — Columnstore is different and should be maintained differently — rebuilding indexes gets prescribed as a cure for performance issues of all types. Sudden query slowdown? Rebuild’em! Blocking? Rebuild’em! High ......(more) Audit SQL Agent Jobs - One probably seldom thinks of the SQL Agent jobs scheduled on the SQL Server instance – unless they fail. What if the job failed because something was changed in the job? Maybe you knew about the change, maybe you didn’t. Once upon a time, I was in the ......(more) a monumental migration to sql server 2016 - This post originally appeared in two parts on my personal blog at flxsql.com and has been reposted here by request. A bit over a year ago, I blogged about my experience migrating a test SQL Server instance from a VM to a physical machine with a little ......(more) Enumerating SQL Server instances on a remote Windows host - Programmatically enumerating a list of SQL Server instances from a remote Windows server can be challenging. If you have the Remote Registry activated, and know the name of the target server, you can fairly easily... ...(more) Is The Server Slower Than It Used To Be? - Economy Of Words When someone says that something is slower than it was before, whether it’s a query, a SQL Server, a website, or an app screen, it’s almost never while the perceived slowness is happening, nor is it reproducible (especially when a consultant ......(more) Extended Events: Detect Implicit Conversions that affect execution plans - Implicit Conversions – Introduction In this blog post, I’ll explain and demonstrate how to use Extended Events to detect Implicit Conversions that affect execution plans on your SQL Server instance. But before we even start with Extended Events, let’s ......(more) SQL SERVER – FIX: Msg 8180 – Statement(s) Could not be Prepared. Deferred Prepare Could not be Completed - While running a linked server query, I encountered an error and learned something new. In this blog we would learn about how to fix error – Msg 8180 – Statement(s) could not be prepared. Here is the complete error message which I received. OLE DB provider ......(more) The Importance of Database Compatibility Level in SQL Server - Prior to SQL Server 2014, the database compatibility level of your user databases was not typically an important property that you had to be concerned with, at least from a performance perspective. Unlike the database file level (which gets automatically ......(more) Evaluating SQL Server System Resource Governor Workload Group "Maximum Memory Grant %" for Adjustment - I'm looking at a SQL Server system with 4 NUMA nodes, 18 cores on each NUMA node. Total RAM: 2 TB.We can easily verify that the SQL Server default workload group is the main CPU consumer on the server.I like to trend active requests and parallel threads ......(more) Let’s talk about trace flags - One of the most confusing aspects of SQL Server configuration is often trace flags. There are lots of trace flags out there, and while many of them are documented, when to use them and when not to is not always clear. The primary reason for this post ......(more) Different Query Store Settings for a Database in an Availability Group - Last week there was a question on #sqlhelp on Twitter about the status of Query Store for a database in an Availability Group. I’ve written about Query Store and Availability Groups before so if you’re not familiar with QS behavior in an AG, check out ......(more) Fixing Your SQL Server: Prioritizing Tasks - Gym Shorts If you said to me “Erik, I wanna run a mile in 10 minutes”, my first question would be “how fast can you run one now?” Most everyone wants their SQL Server to go faster, but getting it to go faster isn’t always the first thing you should worry ......(more) |