Summary Experience Education Services Download CV

Archive for the ‘Responses to Articles’ Category

“Systems thinking” and diversity of digital expertise

Posted by

I recently came across an extremely insightful blog post that articulated a view that I’ve held for years now, and is crucial for forward movement in the digital marketing industry.

Systems Thinking and the current model

Systems thinking is a way of trying to comprehend and optimize the complex nature of how a business operates. Having worked at 6 digital agencies, I understand the power of applying this way of thinking to how you approach creating efficiencies. Most digital agencies have a few brilliant, innovative, and inspiring leaders with a strong vision. They lead by example and try to incite change in the way their business operates from the top down. For smaller shops where there may only be 10 or 15 folks on the floor, these leaders are able to implement change and improvements quite easily with a simple email.

But let’s think about this in the context of an agency that spans the full gamut of Creative, UX, Development, Media/Social, Ad Ops, and Analytics. If you have an account big enough to support relationships with all of these departments, you’re going to need quite a few folks. I’m going to focus my discussion on the Media side of things as that’s my area of expertise.

When a brief comes into the office, generally you have a senior level account or media resource who is ultimately responsible for delivering results. The issue with this is that from brief to final report, execution involves 10-20 different individuals internally and externally for a decent media investment. These individuals can sometimes span across 5-10 different organizations, including your agency with it’s silo’d media, ad ops, and analytics resources. The mid-junior media folks don’t know crap about analytics technologies. The mid-junior analytics resources don’t know the ins and outs of the ad server ecosystem. The ad ops folks don’t understand the business priorities of the client. Your senior resource may (but usually does not) understand all of the mundane intricacies of these crucial moving parts. But they certainly don’t have time to precisely orchestrate a cohesive execution because you’ve allocated their time across 4 accounts, and your account’s business leaders haven’t been able to approve a final communications approach until 4 weeks prior to launch. In the end, no one has time to do anything but react in the best way they can, point fingers at others when they encounter problems that effect their stop on the assembly line, and spin the final results to paint a picture of success.

This system is broken. It’s broken on the client side, it’s broken on the agency side. There are leaders trying to change it from the top down, but it’s a futile effort.

How do we fix the system?

Bottom up thinking and the strength of your diversity

The first part of the solution is changing “top down” thinking to “bottom up”. Some innovations that will impact your business’ bottom line will come from your seasoned Directors. But no one is capitalizing on the dramatic impact that can come from your new hires, interns, and trainees. We need to stop thinking about business decision makers as only CEOs, VPs, and Directors. All of your employees should be business decision makers. Why bring someone on the payroll if they aren’t going to contribute to the improvement of your business operation? Your leaders should in turn provide the vision of future operations, and be scrupulous facilitators; pulling the necessary strings to fast track the innovations and optimization ideas that come up from the bottom. Managers should be constantly encouraging and requiring dialogue across department lines, and between levels.

It turns out that even though your leaders can provide some of the revolutionary process changes that set industry standards, your greatest opportunity for garnering these ideas is to mine the full breadth of experience and expertise of your employees. One Director may bring 15 years of experience from 4 agencies. But 5 junior employees can bring 15 years of combined and varied experience from 10 agencies. This is the true unrealized power of most shops and big agencies

Creating a safe environment for healthy uphill criticism

Another challenge in capitalizing on your diversity of expertise is the need for a truly open and transparent environment. We Americans, we’re trained from pre-school that when we leave home and go out into the world we’re supposed to hold our tongues and do as we’re told. Anything else jeopardizes your future oppurtunities. This mentality works well for big corporations who need workers, not thinkers. The digital industry is moving into it’s adolescence. More and more, agencies and their clients are correctly realizing that advanced tools and big data don’t mean anything without people who know how to use them. We’re at a critical turning point where a generation who’s grown up with technology is coming into a work-force managed by a generation who grew up with vinyl, the walk-man, and home phones.

Here’s the rub, the folks currently at the top will never have the experience of working their way up from a digital intern, having had ground level exposure to things like Doubleclick, Google Analytics, Omniture, or even Pivot Tables in Microsoft Excel. There isn’t anything wrong with that– these tools didn’t exist 20 years ago when they came to the marketing world. But it means that Directors and Managers may think they understand, but really don’t have a grasp on what working with these tools on a tight timeline is like, or the issues that are caused by client demands. Nor can they understand how geeky boring details have major consequences for a client’s ability to get clarity.

These two circumstances– the younger generation’s new skill requirements and resistance to speaking out– mean that the famous “open door” policies simply aren’t effective. This puts the onus of fixing an organizational problem on junior employees, not Managers and leaders. I’m not implying we should get rid of these policies, but rather build upon them; make them multi-directional. Don’t wait for your employees to come to you with issues. If you’ve done a good job recruiting, you should assume that everyone has a few things they might want to change. Don’t wait for them to build up the courage to tell you that you’re doing something wrong, set up a private meeting with one junior Ad Ops. Take them off-site if you can spare 45 minutes. Plead with them that they individually are extremely important to your business, and ask them if they have any thoughts on what you’re doing wrong. They may not bring you a ground-breaking idea on this first approach, but by establishing a personal connection, you’ll be giving them permission to come back later when that industry changing big idea comes to them.

Understanding and addressing everyone else’s system

Now you have a process in place to constantly improve your own system. However, your system is actually just one tiny part of the ad industry eco-system. I’ve focused on the digital media side of things. If you’re working on a typical account, you also partner with one or more creative agencies, maybe a traditional media agency, a PR agency, and that’s just at the client level. For your own work, you have maybe 100+ publisher relationships, technology vendors, brand study vendors, etc. And of course, there is the client themself.

Each one of these partners has their own system and none of them thinks outside of it. If you want to deliver successful initiatives for your client, you’re going to have to think outside of your system and optimize the way you work so that assets, information, and money is smoothly flowing back and forth. This may be difficult as some of these “partners” are actually competitors plotting behind your back on how they can steal your piece of the business. But this is never going to change. We must not be so preoccupied with keeping our accounts that we aren’t constantly striving to take risks that result in better value. As an industry, we have to start keeping our accounts loyal by providing a level of thought leadership, service, and efficiency that outshines a competitor’s ability to under bid you. And agencies are going to have to start being nicer to each other. Total ad spend, driven by constant growth in Video, Display, RTB, and Mobile Social will all provide enough business to ensure that there still won’t be enough agencies or employees to keep up with business needs over the course of the next few years. If we can all just accept that there will be dollars for us, we can start the process of streamlining how much it costs us to operate our businesses. That is what will ensure your continued growth and profitability.

We’re also going to have to start being braver with how we deal with our client managers. If a timeline is agreed to, restrictions signed off on in a scope of work, or a request is not feasible, we’re going to have to start say no more often. It’s your client’s job to make sure that they understand their options and timelines and communicate that back to their bosses. It is not an agency’s job to deliver an 8-week long process (that you’ve now refined and optimized) in 3 weeks because your client contact doesn’t know how to make it clear to their boss that 3 weeks is not reasonable. It’s a trap trying to cut corners and push your teams to deliver things too quickly. Doing this ALWAYS results in a campaign, reporting data, and analysis that aren’t up to the standards we should hold in our industry. Rome wasn’t built in a day, nor should any solid media plan. If your client is trying to do something like re-position their brand or raise awareness, you better be clear that if they actually care about accurate, measurable results rather than board room anecdotes, they’ll have to give you time to figure out all the necessary details

At the end of the day…

whenever you’re on a phone call, in a meeting, or with a managee– we have to start keeping in mind that we’re handing off information and insights into someone else’s system. Everything we receive or send gets sent and received another 6 times and is processed 12 times along the way. We have to get better at thinking about deliverables and results in terms of cycles and final destinations. Not simply direct contacts and immediate needs. We have to communicate better and stop pretending that other people’s priorities and issues aren’t as important as our own in the bigger picture. And we have to start doing this together rather than pressuring each other or undermining the honest efforts of our peers.

Do Display Ads Influence Search?

Posted by

I was a psych major. For those of you who wondered what us “soft” science Psych majors studied in college– I was basically trained on how to call BS on any claim, statement, or inference of causality. Why? Because measurement in Psychology is not like measuring mass or temperature. It’s an art. It’s complex. And 9 times out of 10, your causal conclusions are flawed by some confounding factor you haven’t thought of. This is the basis of my obsession with Digital Media, and my long-time frustration with the oversimplification of Performance Analysis of Display media campaigns.

Because I have nothing better to do than read 30 page papers about Display and Search Media Attribution, I wanted to share with you all a detailed criticism of the Harvard Business School “Do Display Ads Influence Search? Attribution and Dynamics in Online Advertising” working paper that was published a couple of months ago. My hope is that some day, my fellow Digital Media professionals will one day accept the fact that campaign results and analysis for display campaigns will never be appropriately understood or accurately summarized in a one sentence executive summary. As long as we continue to condense the complexity of campaign analysis into quick sound bites and talking points, none of us will ever actually understand the full picture of the media mix.

First, the overall message I take away from the paper is clear and valid, “managers should carefully consider the interaction and dynamic effects of search and display advertising”. You should know this already. And if this is the first time you’ve heard this, the paper is a great place to get a deep dive into the justifications for this statement. It’s an important message that is often forgotten in the madness of the corporate game and I’m glad they’ve called this out.

Now, on to my pain points…

A huge problem with the generalization of their conclusions lies in their data. It’s all in the dirty details at the bottom of page 7. Their whole data set relied on “Using Internet Cookies”. I feel that a deeper understanding of the ad serving ecosystem begs the question if this is a valid data collection methodology for the kind of advanced statistical analysis they are performing. The paper makes several assumptions by making this statement:

  • Cookies are reliable – This is not nor has it ever been true. Reporting what the researchers would call “standard metrics” using cookie methodology is fairly reliable and great for directional insights, general campaign auto-optimizations, and a top level view of success. But they are not a reliable data source for the type of quantitative statistical analysis that is being performed in this paper. Doubleclick For Publishers (DFP) clearly states “it is common to see campaign variances of up to 20%”. I think it’s conceivable that people less likely to be cookie’d could correlate with a variable that dramatically effects the campaign results. For example, people who access desktop sites through non-home computers are less likely to have a cookie track activity past an impression, and might be more likely to be served an impression that is not tracked. Could this group have a higher conversion rate for some confounding reason?
  • The entire population only used one desktop computer during the campaign - This data does not account for users who, for example, were exposed to a display ad on one device, and completed an application through an organic or direct site visit through a second device or location. This is one of the many reasons understanding effects of display ads on brand awareness or purchase intent is crucial for measuring display ROI.
  • All conversions took place on a desktop – Does this bank have a mobile app or site? Is that part of the measurement methodology? Users could have similarly navigated to the banking website on an iPhone after having been exposed to a Display Ad on a desktop. Also, the study never outlines whether mobile search or display placements were a part of the data set. These ad impressions are much less likely to track through a conversion from an impression. Since mobile smart phones now account for half all mobile phones, a large group of people have the capability of being exposed to a banner ad impression without marketers being able to attribute it to a conversion. The authors themselves mention that their study does “not consider spillover effects of search and display into [mobile and offline]“.

Aside from the reliability of cookies, the agency attributed any post impression conversion within a 30 day look-back window fully to Display (a common but misguided practice) except for Paid Search. But only because there was no technical solution in place to de-dupe attribution from Paid Search (a la Doubleclick Search). This is another questionable oversight in their data which, in my opinion, calls to question whether the authors’ directional insights on SEM vs. Display media investment in the empirical environment are valid in the real world. I believe the answer is no. There are simply too many user cases where a consumer can be influenced by a display ad impression before a paid search click without an ability to measure the overlap or the value of each channel’s influence.

The paper goes on to describe a series of statistical tests that I had to pull out my college stat text book to get through. I don’t find anything unsound in these methodologies, nor is evaluating the statistical methodologies something I’m attempting to do in this criticism. Rather, I’m bringing hands on knowledge of the ad delivery and reporting methodologies into the context of this study. In display and search measurement, there are many holes in the data and inaccuracies caused by technical malfunctions in the transmission of ads or cookies as well as human error in technology implementation. Without a robust exploration of the possible effects of these important variables, I don’t feel the statistical analysis provides results that should be used to make business decisions blindly from other sources of analysis like brand surveys, focus groups, or cross-channel analysis. However, I believe it provides extremely valuable insights in the way of questions that marketers should ask themselves when trying to understand the attribution model for their specific product or brand.

©2024 J Reese – Digital Media Resume