Project: Auto Blink

Mu He | Apr 3, 2024 min read

If you’ve ever woken up to an unexpected parking ticket because you forgot to move your car, you’re not alone. Municipal parking restrictions, especially overnight rules, can sneak up on anyone. We tried multiple methods to remind ourselves to move cars, and we found a few effective ways to avoid these annoying fines and keep our mornings stress-free. In this post, we’ll evaluate several effective ways and my ultimate solution - Auto Blink.

1. Mobile Phone Reminders

In my case, I use the shortcut app on my iPhone to create a reminder that triggers every night at 10pm. Here is my recipe:

  • Vibrate device
  • Turn up the volume
  • Speak “Time to move your car”
  • Pop up an alert “Do you want to cancel the reminder?”
  • If not, then open “Ring” app

Pros:

  • Low tech and easy to implement

Cons:

  • It is easy to snooze or ignore notifications

We have a Ring flash camera on our driveway, so when the app pops up, I can see the video of the driveway and make sure the car is moved. With this method, we only got 1 ticket so far which is much less than the number of tickets we used to get.

2. Ring

With the development of AI, we can now use machine vision to monitor the driveway and alert us when there is a car parked there with relatively low cost. We already have a Ring camera on our driveway, so I decided to give it a try. Kudos to Dusty (@dgreif) for the great project ring, it is a simple and easy to program to interact with Ring’s API. I have managed to get it running on my local server. HOWEVER, to capture the refresh token all the time (to avoid 2FA for auto-login) is a bit of a pain. You may find more details in this wiki. Therefore, I decided to give up on this method.

Don’t ask me why, but we do also have a set of Blink cameras in our house. I have been using them to record our home and see what is happening when we are not around. They are not the best quality, but they are cheap and cheerful. Thanks to the contributors for the blinkpy library, I got the program running again on the Blink cameras. Fortunately, the authentication for the Blink cameras is much “user friendly” than Ring’s. The script is very simple, it will call the API to get the camera screenshots, then save the to a folder.

await camera.snap_picture()  
await camera.image_to_file('/local/path/for/image.jpg')

After that call the OAI 4o-mini API to detect if there is a car on the driveway, and if my other car is in the garage. I have set up the email alert in the script as well. So if the car is not on the driveway, my wife and I will be notified immediately.

def encode_image(image_path):
  try:
    with open(image_path, "rb") as image_file:
      return base64.b64encode(image_file.read()).decode('utf-8')
  except Exception as e:
    sentry_sdk.capture_exception(e)
    raise e

def get_gpt_response(timestamp, camera_name):
    try:
        client = OpenAI()
        base64_image = encode_image(f'''./data/{timestamp}_{camera_name}.jpg''')
        
        camera_config = {
            'Garage': {
                'question': f"Is my {CAR_NAME} in the garage?",
                'not_present_message': '{ALERT_MESSAGE}',
                'present_message': '{REASSURED_MESSAGE}',
            },
            'Front': {
                'question': f"Is my {CAR_NAME} on the driveway?",
                'not_present_message': '{ALERT_MESSAGE}',
                'present_message': '{REASSURED_MESSAGE}',
            }
        }

        if camera_name not in camera_config:
            sentry_sdk.capture_message(f'Invalid camera name: {camera_name}', level='error')
            raise ValueError(f'Invalid camera name: {camera_name}')

        config = camera_config[camera_name]
        messages = [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": f"{config['question']} Only respond with 'YES' or 'NO'. Please be very confident in your response. If you are not confident, respond with 'NO'."},
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": f"data:image/jpeg;base64,{base64_image}",
                        },
                    },
                ],
            }
        ]

        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
        )

        result = response.choices[0].message.content.strip().upper()

        if result not in ['YES', 'NO']:
            sentry_sdk.capture_message(f'Unexpected GPT response: {result}', level='error')
            raise ValueError(f'Unexpected GPT response: {result}')

        if result == 'NO':
            sentry_sdk.capture_message(config['not_present_message'], level='error')
            send_email('{EMAIL}', config['not_present_message'], config['not_present_message'])
        else:
            sentry_sdk.capture_message(config['present_message'], level='info')

        return result

    except Exception as e:
        sentry_sdk.capture_exception(e)
        raise

At last, I packaged the script into a docker container and run it on my local server. The whole system is now running automatically every night at 10pm.

This method certainly works with other cameras as well. You can use any camera that supports RTSP stream, and any LLM model that you like. Also, you can easily extend it to more scenarios like checking if the garage door is open, or if the water pipe is leaking, or you want to create a timelapse video of your garden or backyard. Moreover, you can leverage the motion detection feature of the cameras to trigger the LLM API call when Amazon delivery trucks show up at your door and greet the delivery person.