Welcome to Internal Tech Emails: internal tech industry emails that surface in public records. 🔍 If you haven’t signed up, join 60,000+ others and get the newsletter:
Mark Zuckerberg on social issue research
From: Mark Zuckerberg
Sent: Wednesday, September 15, 2021 9:31 AM
To: Javier Olivan; Alex Schultz; Danny Ferrante; Chris Cox; David Ginsberg; Nick Clegg; Joel Kaplan; Jennifer Newstead; Guy Rosen; Andrew Bosworth; Sheryl Sandberg
Subject: Social issue research and analytics -- privileged and confidential
Recent events have made me consider whether we should change our approach to research and analytics around social issues.
Apple, for example, doesn’t seem to study any of this stuff. As far as I understand, they don’t have anyone reviewing or moderating content and don’t even have a report flow in iMessage. They’ve taken the approach that it is people’s own responsibility what they do on the platform, and by Apple not taking that responsibility upon themselves, they haven’t created a staff or plethora of studies examining the tradeoffs in their approach. This has worked surprisingly well for them. Instead of being harshly criticized for not doing anything to fight CSAM historically, we have faced more criticism because the fact that we report more material makes it seem like there’s more of that behavior on our platforms. Interestingly, when Apple did try to do something about CSAM, they were roundly criticized for it, which may encourage them to double down on their original approach.
YouTube, Twitter and Snap take a similar approach, to lesser degrees. YouTube seems to intentionally bury its head in the sand to stay below the radar and not be the center of attention. Twitter and Snap may just not have the resources to do this kind of redearch.
I think we should be commended for the work we do to study, understand, and improve social issues on our platforms. Unfortunately, the media is more likely to use any research or recommendations produced to say we’re not doing everything we can (implying for craven purposes) rather than that we’re taking these issues more seriously than anyone else in our industry by studying them and looking for solutions, not all of which are reasonable to implement because everything has tradeoffs.
Leaks undermine our ability to do this work in a way that isn’t incredibly destructive. This may be part of why the rest of the industry has chosen a different approach towards these issues.
Further, when we’ve tried to enable independent academic research -- way beyond anything the rest of the industry is attempting -- it is questionable whether execution errors undermine our efforts and destroy more value than is created by the research we’re enabling.
Given these realities, I’m interested in recommendations for how we might adjust our research and analytics programs going forward.
From: Javier Olivan
Date: Wednesday, September 15, 2021 at 10:48 AM
A couple of thoughts since this has been a controversial topic for many years. There are two very different areas here:
External research:
We should not live under the illusion that there won’t be mistakes or plenty of areas to criticize. Missing rows in this data set was a dumb mistake, but there are plenty of really hard tensions to square between user privacy / value of the data sets / who gets access to them, etc… so if the media wants to pick on it (vs. see the best effort side of it) / assumes bad intention, this will always be a fruitful area to write plenty of negative stories.
Given that – we should decide whether it is a net positive or not. Nick made a compelling case on why it is still better on net, but I want to make sure we are all clear that there will be no lack of ‘easy to criticize’ limitations on the data sets we open / mistakes will happen.
Internal research
Leaks suck, and will continue to happen, unless we find a way to eradicate them.
Given that – is it still worth trying to understand these issues? I think it is the responsible thing to do / I would love for us to continue trying to understand how can we make our products better for everyone, but maybe we should limit the surface to those areas where we at least see some clear degree of correlation between usage of our products / the specific issue.
[This document is from California v. Meta (2026).]
X link
Threads link
Previously: Mark Zuckerberg’s WhatsApp messages (October 6, 2021)
Previously: Mark Zuckerberg: “Please Resign” (September 22, 2010)
Previously: 35+ documents from Mark Zuckerberg in the full archive
Inside The Tech Emails Files
See how the biggest companies in tech actually operate — in their own words.
The Tech Emails Files are 300+ internal documents pulled from court filings we track year-round. Strategy memos. Board emails. Messages between CEOs and execs at Apple, Google, Meta, Microsoft, OpenAI, Tesla, and more.
Investors use it to understand how leadership actually thinks. Journalists use it for primary sources. Founders and operators use it to study how the biggest companies make decisions.
Upgrade to a paid subscription, and you’ll unlock access to the full archive:
Mark Zuckerberg on Cambridge Analytica
From: Mark Zuckerberg
Sent: Monday, January 30, 2017 8:10:16 AM
To: Javier Olivan; Alex Schultz; Rob Goldman; Danny Ferrante; Mark Rabkin; Andrew Bosworth
Subject: Cambridge Analytica
I’m seeing more articles like this one come out claiming the Trump’s campaign used FB and analytics in very precise ways that go beyond what others have done before:
https://motherboard.vice.com/read/big-data-cambridge-analytica-brexit-trump
I can’t tell how much of this is just a dramatic telling the best practices of marketing today seen through the lens of survivorship bias, or how much of what they’re done is truly novel and interesting work, at least in the field of politics.
Can someone explain to me what they actually did from an analytics and ads perspective, and how advanced it actually was?
From: Rob Goldman
Sent: Monday, January 30, 2017 9:06 AM
I took a pretty close look at the trump account just after the execution. Nearly all of their spend (97%+) used custom audiences. As a result, it will be hard to know precisely how they arranged people into audiences since they did it on their side. At a minimum though, they must have already had a personal identifier (most likely email) to match to on their side.
I suspect that this article overstates the case at least slightly since 33% of their spend used lookalikes, which is our targeting option for statistical similarity. That would suggest that their coverage was low and they needed us to expand it for them.
Adding Cheryl from ads. Someone on her team might be able to take a closer look at the campaign naming conventions to see if their logic is apparent.
From: Cheryl Dartt
Date: Monday, January 30, 2017 at 9:42 AM
We took a high level look at spend and ads effectiveness during the lead-up to the election, but didn’t dig deeply into sophistication of techniques employed. Let us do a bit of looking into the campaign structure and practices and will summarize what we see.
From: Andrew Bosworth
Sent: Monday, January 30, 2017 9:58:03 AM
Thanks Cheryl, I suspect the results from your deep dive will be broadly interesting either way. Don’t limit yourself to CA and their work but let’s look at the campaign holistically.
My general understanding is the Trump campaign actually took all the advice we gave unlike Hillary. They used all our tools. That *is* surprisingly novel as most advertisers have a “way of doing things” and are slow to adapt but speaks more of our platform than CA. A few examples:
They ran tens of thousands of creatives and iterated quickly to find what was working, further sub segmenting audiences (using custom audiences) based on response.
They used different formats early but found video was most effective so were using that primarily by the end (90+%)
They were primarily fundraising on Facebook, which means rallying the base instead of trying to win over marginal voters. That was an effective strategy that lead to very low CPMs and makes iterating to find market fiit for your message much easier. The rumor is that they turned tens of millions spent on FB into hundreds of millions for the campaign, which they spent elsewhere.
I’ve heard separately that their entire electoral strategy was to rally the base rather than send a broadly appealing message. Few marketers are interested in doing that so it stands to reason they would have unusual results.
Net net, my guess is that CA did have to do some interesting work to quickly take results from campaigns and spin them into the next, and the campaign had to do some interesting work to generate so much creative, but I’m skeptical that Cambridge Analytica has any data or expertise that couldn’t be replicated readily by someone else. I’m suspect we will find that the majority of the advantage was using our platform to the fullest.
boz
From: Mark Zuckerberg
Date: Monday, January 30, 2017 at 10:20 AM
Interesting. I’m curious to see what your analysis shows.
Other anecdotes have come out over time like that they used analytics from social media to determine locations for rallies, etc. Would that be using ad tools too?
From: Andrew Bosworth
Date: Monday, January 30, 2017 at 10:26 AM
I have heard that rumor also. They certainly could do so. Run a large campaign, see if there are geographic similarities in response rate, plan rallies there. I think we would need inside info to confirm or deny. +Fischer as our political sales team supporting them might have more insight also.
boz
From: David Fischer
Sent: Monday, January 30, 2017 8:50 PM
Boz’s analysis is largely on the money. I reached out to Annie Lewis, who manages the FB politics team that worked with the Trump campaign. She’s cc’d here if you have follow up questions. Here’s her response to the article Mark forwarded on CA:
The Trump campaign had a significantly smaller media budget ($200M compared to $300M for HRC, not even counting the additional Clinton super pacs.) This budget reality drove a focus on digital simply because it was cheaper, and then to Facebook, where they could do the most testing and determined the highest impact. They also looked at Facebook through the lens of a non-political advertiser; that is, they did their diligence in researching what other top performers did in other sectors like gaming and applied that here. And, they ultimately did exhibit more willingness to test than any other political client we have – largely due to the fact that the campaign itself had so little infrastructure at the outset so they had to try new things out of necessity (versus HRC which had been building for a year plus.)
While Cambridge Analytica is the focus of this Vice article, I’d clarify that the majority of Custom Audiences advertising was against Website Custom Audiences for fundraising. The other data Cambridge Analytica targeted was some of their own modeled stuff referenced here, as well as data from the RNC, and as mentioned, their own website custom audiences. I do believe the press around the role of Cambridge Analytica’s data specifically is over exaggerated, due to CA’s interest in telling the story.
More context: a competitive, results oriented mindset drove the culture as well as the decisions. They used our most advanced measurement products for all their objectives (fundraising, persuasion, Get out the Vote), which allowed them to quickly make optimizations against typically hard to measure metrics like voter intent and early/absentee votes. For example, this measurement found that positive, jobs focused ads worked better than the negative anti-Hillary stuff, which then informed other strategies.
Boz mentions the thousands of ad variations - even using two FMPs simultaneously to pit their performance against each other for a few months (at the end of each day, the budget was reallocated based on performance between two FMPs.) We’ve never seen anything like that in Politics and is quite rare across other verticals.
They also used video – as Boz mentions, 90%+ - which drove consistently ROI positive DR results. We’ve never seen video used for DR in that way. On the rallies: it’s possible they did that, but we weren’t involved.
[This document is from In re Facebook, Inc. Securities Litigation (2026).]
X link
Threads link
Previously: 35+ documents from Mark Zuckerberg in the full archive
Mark Zuckerberg: “Winning at messaging”
On 3/22/13 9:07 AM, Mark Zuckerberg wrote:
Being the best messaging service is the biggest opportunity and mitigation to the biggest threat we face today. The space keeps heating up, including Google about to get into this in a big way, Line with more than 1,000 engineers just focused on their app and rumors of WhatsApp expanding into more services as well.
We have a team of ~60 engineers, which is a lot, but given the importance of success here, I often find myself wondering what else we could do to increase the chances of our success.
It seems like a good time for me to put this question to you guys, now that a lot of the big features we’ve worked on for a while are nearing release (eg Chat Heads, broadcast, composer integration, stickers, like button, aura, etc) and we’ll be able to start shipping more Android code post-Ansible release.
If we had more engineers working on messages, what else would we be able to do?
If we could get other teams at Facebook to build anything we wanted for messaging, what would we have them do?
What are the process bottlenecks that we have today that we could improve to speed things up?
I’d love to hear all of the ideas you guys have here, including relatively extreme things that might sound crazy at first.
[This document is from FTC v. Meta (2025).]
X link
Threads link
Previously: Mark Zuckerberg on acquiring Snapchat and WhatsApp (October 26, 2013)
Previously: Mark Zuckerberg emails WhatsApp cofounder (April 6, 2013)
Previously: Mark Zuckerberg: “Ship the app” (September 11, 2011)
Previously: 35+ documents from Mark Zuckerberg in the full archive
A note from @TechEmails
Every year, I track hundreds of court cases and review more than 10,000 filings to bring you Internal Tech Emails. If you like @TechEmails, and would like to help make this work more sustainable, consider upgrading to a paid subscription.
You’ll be supporting the research that drives Internal Tech Emails, and will help ensure that it can continue publishing. And you’ll also receive access to the full archive of internal tech emails, with 300+ documents from Apple, Google, Meta, Microsoft, OpenAI, Tesla, and more.
Thanks for reading!
-Internal Tech Emails
Sent from my iPad