So much easier to do much of this now, compared to implementing this a few years ago: https://doi.org/10.1016/j.cviu.2019.07.010
That used BigTable and multiple buckets to approximately search the vector space. Now you can use Google Cloud Vertex AI Matching Engine: https://cloud.google.com/vertex-ai/docs/matching-engine/overview