From the European Union's General Data Protection Regulation (GDPR) to Congressional hearings on how fake
news and propaganda influenced the 2016 US election, the importance of social data governance has become
part of the media narrative. As data engineering professionals, how can we ensure personal data is protected
while also helping analytics firms derive insights from social data for critical business functions in the world's
largest companies?
Revelations of data breaches
Revelations of data breaches have led to a mindset shift amongst enterprise technology professionals. Until
now, conventional wisdom was to amass "data lakes", hoarding every bit of information that might someday
prove relevant or useful. But good data governance means ensuring that data is managed in accordance with:
– Retention and expiration policies that purge outdated data
– Respect for user actions such as edits and deletes to their data
– Contextual permissions for specific use cases
– Policies requiring redacted or hashed metadata
– "Differential privacy" techniques such as quantized statistics to protect small audiences
As more enterprise functions begin to employ their own data science teams, data sharing within and across
organizations is becoming more common. Data governance policies and procedures will only become more
important and complex. Recent incidents have shown how open APIs have compromised user data in irrevocable
and unforeseen ways. Over time, data collaboration will require better safeguards.
At Socialgist, we provide a variety of mechanisms which help customers get the right data, and maintain it over
time. Whether by using feeds of user edits and delete events to continuously update your dataset, or Search API
access that lets you retrieve refined datasets with complex queries, right-sizing your social data footprint is more
important than ever. We're eager to continue to improve our platform's data governance tools for customers as
best practices continue to evolve.
