Sunday, October 13, 2024

Polls are a 20th century tool in the 21st century

    In the information age you cannot honestly get a representative polling sample of anything other than "people who are willing to take surveys". People who take surveys are weird. I know this because I take surveys. And after college the only job I was able to get was to sit in an Abt SRBI call center in Hadley Massachusetts where I had to conduct Pew Research surveys asking women about their pap smears; the more honest answer as to why I take surveys is that I have a penance to pay.

    Accuse me of using the availability heuristic. Having worked in a research call center, I can tell you that the people who take these surveys are not reflective of average Americans. They are people who still have a landline phone. They are the people who are so lonely, bored, and senile that they will send thousands of dollars in Google Play gift cards to scammers with foreign accents. They will talk to anybody that will listen. You'll save more lives doing survey calls than you will working the crisis line, believe me, I've done that job too. 

    In the 21st century some surveys are still only conducted on landlines as a way to guarantee that the person that is being rang is physically in the place that they are being surveyed. Think of all the people you know that have a landline and actually answer it. All of the people you know that's had a Nelson Box in their house. Do those people, even if you actually even know any, reflect an accurate sampling of America?

     The second cohort of data you're getting in your polling is bad actors. There's nothing stopping an individual from using tools like Selenium and Beautiful Soup in conjunction the Chat GPT API to automatically fill out surveys online. Heck, you can do the same exact thing on phone lines. Here's a quick example I whipped up using an AI vendor:



For what it's worth, I time-boxed for myself an hour to configure and coach the LLM as well as record the audio. With a little more time this LLM could do much better. This kind of work is literally not worth any one individual's time outside of tinkering. But if someone has a vested interest in manipulating polls? This is money on the table.

   While individual bad actors do exist, they typically represent a smaller portion of the overall sample. It wasn't unusual for someone on the phone to respond to your questions while being completely unserious or dismissive. I remember an older colleague once sharing a story about how "sometimes you just do what you have to do to get the job done", which ended with her telling me about a time where she got a guy to finish a whole survey even though she could tell that he was pleasuring himself on the call. The most you could hope for is to note that the survey taker was disingenuous or likely messing with you, but that was it. It wasn't the survey taker's jobs to triage the data, however when's the last time you saw a public election poll that listed the sample size and notes on why certain responses were excluded from the data?

    The absence of clear information about research methodologies in most election polls suggests that these polls are more of a media-driven product designed for public consumption than a reliable source of objective data. This lack of transparency turns polls into a form of entertainment or narrative shaping, where their primary function may be to engage audiences, influence perceptions, or reinforce certain viewpoints, rather than to provide rigorous and unbiased insights into voter behavior.

Further Reading: The Problem with polls.


Sunday, June 23, 2024

What I've Read Halfway Into 2024

The Woman in Me - Brittney Spears

 

This book made me realize just how in the limelight Brittney Spears has been in almost my entire lifetime, because I realized I was aware of just about every aspect of her life as I read through this book despite never really being a fan. This book is heartbreaking.
 

Kill It With Fire - Marianne Bellotti

 


I bought this book solely because of the cover and identifying the dumpster fire on the cover as being the place I was working at the time. It was happenstance, but this book is really good. It provides a framework for dealing with technical debt, offering strategies to update aging systems without massive resource expenditure in a way that I could identify with personally. It addressed large technical problems/projects I have faced at work and confounded solutions and approaches to solving these problems. This book helped me land my next job and escape the dumpster fire. 

 

 

Flow My Tears, The Policeman Said - Phillip K Dick

 


I’ve read four other Phillip K Dick books and a handful of his short stories. I don’t know what I can say about this book without giving it away but this was likely my least favorite of his books that I have read so far. I'd suggest reading first Do Androids Dream of Electric Sheep and Ubik.

 

 

Fighting Fantasy Books - Various Authors
 


I came across these books when looking for solo RPG books; seeing that Steve Jackson had written and co-authored some of these got my attention. I read / played City of Thieves by Ian Livingstone and Sorcery! - The Shamutani Hills by Steve Jackson. These books are Choose Your Own Adventure books with dice rolling. At first I thought that these books were a lot more forgiving than the CYOA books, but by them time I finished them I realized the game-play is exactly the same: if you don’t make the exact right decisions in the right order, you will not win. Also like the CYOA books, these are geared towards children. I would recommend Ironsworn instead of these books. 

 


Neuromancer - William Gibson


is the first book I’ve read since I can remember where I had to use Sparknotes to keep up with what was going on. The characters speak in a casual way to each other, similar to hard boiled detective novels which is tricky (for me) to follow. There’s a lot going on here; you can see where the Wachowskis got a lot of their inspiration from for The Matrix. It’s fascinating how Gibson hit the nail on the head in the 80s when it came to the evolution of AI and technology. It gets said a lot already but it really needs to be acknowledged just how prophetic Gibson was when it came to technology here. 

 


The Medium is the Massage - Marshall McLuhan

 


The central idea of this media analysis book is that technologies are the messages, rather than the content of the communication. I know that's a heady statement to make, but that's what this book is. It mainly relies on formatting and images to convey its message. Many times while reading this book, I kept thinking of the “It insists upon itself” meme; I do think that this book insists on itself. I will also say that there are a lot of cool ideas in this book that, if you were a communications major or marketing major, you could write papers on and elaborate on. You can also chase down and explore further the many ideas that come up in this book.

She Come by It Natural: Dolly Parton and the Women Who Lived Her Songs

- Sarah Smarsh

 


My friend lent me this book after I said I wanted to read more book about and by women. She Come by It Natural is a heartfelt tribute to Parton and the women who resonate with her music, but it doesn't bring much new to the table when it comes to being a biography about Parton. Smarsh's admiration for Parton is clear and her narrative is filled with personal anecdotes and reflections and the book largely reiterates well-known aspects of Parton's life and career. 


I was under the impression that most of America knew of Parton and the charitable work she has done. On the subject of Parton’s Imagination Library project, the book quotes children’s author Robert Munsch saying “I thought of Dolly Parton as this singer with the really big boobs who was in the movies with, like, the really big boobs. I had no idea.” I was reading this book while riding the bus and after asking what I was reading a lady said “I used to LOVE watching Elvira on TV”. I guess if that's you're impression of Dolly Parton, maybe check this one out.

 


Empire of The Summer Moon - S.C. Gwynne


Gwynne's meticulous research and vivid storytelling bring to life the incredible saga of Quanah Parker, the last great Comanche chief, and his extraordinary journey from a fierce warrior to a respected leader and eventual advocate for his people. The book not only provides a comprehensive account of the Comanche's way of life but also paints a broader picture of the American West during a time of profound transformation. Gwynne's ability to blend historical facts makes this a compelling read that is both informative and captivating.

 

Growing up in New England and learning about the tribes that lived there versus the plains tribes which I really only first really learned about in this book was fascinating. Understanding how torture, murder, and rape were accepted parts of plains natives' lives really helps shatter the Noble Savage myth that a lot of us are taught. I've listed these books in the order in which I finished them. I read half of this book during lock-down, and recently finished it. I would recommend this book before all others on this list.

Thursday, February 29, 2024

How to exclude traffic from Google Analytics

With previous versions of Google Analytics (ga.js) you could simply add a utm_nooverride=1 parameter to a redirect URL to retain the source data from the first referral instead of attributing conversion to the last redirect. If you are trying to exclude online payments / checkout data from Google Analytics: This doesn't work with redirects to banks as this parameter is not passed along, so the issuer will be seen as the referrer.

In the most recent version of Google Analytics library at the time of writing (analytics.js), the utm_nooverride=1 parameter is not a recommended (and perhaps unworkable) solution (as noted in this blog here). The recommended solution in Universal Analytics is to add the domains to the referral exclusions list. From Google's own documentation:

 

The referral exclusion list uses CONTAINS matching. For example, if you enter example.com, then traffic from sales.example.com is also excluded (because the domain name contains example.com).

 

When you are adding domains to your exclusions list, you can add the bank url base domain. By adding the base domain, you can exclude subdomains of redirect referral URLs (and 3DS redirect URLs for payments). Please note, this solution does require some upfront manual work, but will exclude these redirects as referrals in the future.


There also seems to be some advanced workarounds using cross-domain tracking, which you can find in Google's own documentation. 


Wednesday, February 28, 2024

Why did my expired credit card get processed?

An expired credit card is not necessarily an invalid credit card. It's ultimately your credit card issuer's decision if they want to accept your transaction. While the merchant who processed your transaction almost certainly sent along your CVC code and expiry date in the payment request, sometimes the credit card issuers and/or acquirers do not even look at or process that information. If there is enough information to go off of to for the issuer to believe that this is a valid transaction, they still may decide to process your transaction regardless of your card being expired or the CVC being incorrect.

Most major credit card issuers and acquirers now also offer merchants software that automatically updates subscription customer card data as well. On top of that, expiration dates are not considered sensitive information. Under certain use cases, it's fine for merchants to retry your transaction with a new expiration date - it's pretty easy to guess and build logic around what your new expiration date probably is.

This also means that merchants should never decline a card at checkout just because it looks to be expired. In my experience, there is no change in authorization rates between expired cards and cards that have not expired yet. 

If you are a consumer, always destroy old expired cards when you receive a new one. And do not assume your account has closed just because your credit card has expired.