#
"*": "https://raw.githubusercontent.com/wefindx/schema/master/project/oo-item.yaml"
#
"base:title": "0oo - Postgres synchronization solution"
"og:title": "Postgres synchronization solution"
"og:description": "This is a challenging computer problem, data synchronization but I have an idea I plan to use a inefficient rolling hash and sorting to solve the problem of synchronization. This is similar to rsync"
"og:image": "https://avatars0.githubusercontent.com/u/28134655"
"og:url": "/intent/52001/"
"base:css": "/static/css/bootstrap.min.9c25540d6272.css"
"base:extra-css": "/static/css/base.57997aeac1df.css"
"base:favicon": "/static/favicon.acaa334f0136.ico"
"base:body_class": ""
"layout:logo": "/static/0oo.8d2a8bbef612.svg"
"layout:index": "/"
"layout:menu": "/menu/"
"layout:categories": "/intents/"
"layout:ideas": "/methods/"
"layout:projects": "/projects/"
"layout:users": "/users/"
"layout:about": "/about/"
"layout:help": "/help/"
"layout:bug_report": "https://github.com/wefindx/0oo"
"layout:login": "/accounts/login/"
"layout:light-off": "/darken/?darken=true"
"layout:set-multilingual": "/mulang/?mulang=true"
"layout:lang": "Kalba"
"layout:set-language-post-action": "/i18n/setlang/"
"layout:csrf-token": "2EuAIu4uKV9BPgs8Sm0EMrtRys5519Ltbr6OLFrntCL3qwyqntXdpmf55zh3N4yQ"
"layout:input-next": "/project/52001/"
"layout:languages": [{"code": "ja", "is-active": "false", "name": "日本語"}, {"code": "lt", "is-active": "true", "name": "Lietuviškai"}, {"code": "zh-hans", "is-active": "false", "name": "简体中文"}, {"code": "en", "is-active": "false", "name": "English"}, {"code": "ru", "is-active": "false", "name": "Русский"}, {"code": "oo", "is-active": "false", "name": "O;o,"}]
#
"item:title": "Postgres synchronization solution"
"item:score": -8.0
"item:mission": "I want to take one set of data and synchronize it with another set where there could be changed to columns, new rows and deleted rows"
"item:permalink": "/project/52001/?l=lt"
"item:base-administration": false
"item:body": |
This is a challenging computer problem, data synchronization but I have an idea
I plan to use a inefficient rolling hash and sorting to solve the problem of synchronization. This is similar to rsync
"item:destinations": |
"item:source-date": ""
"item:permalink": "/method/52001/?l=lt"
"item:owner": "chronological"
"item:created": "2022-08-16T13:27:07.939085"
"item:ownerlink": "/user/198/chronological"
#
"item:result:items":
- "id": "t-146001"
"url": "https://github.com/samsquire/postgres-sync"
"text": |
I created a vagrant setup with 3 postgres installations and created a cronjob to run the sync script
"cost": -5.0
"currency": "HUR"
"assets": |
My vagrant setup uses persistent disks and uses ansible to deploy the cronjob and sync script. It is configured by YAML file. I've also installed psycopg2 and I found documentation on how to retrieve the tables in a database in Postgres. It's just a matter of writing the sync algorithm now.
My problem is detecting which side is the winning copy.
When one side changes the data there shall be a different hash and the changed rows are detected. This part I understand.
The problem is detecting which side is the latest change and which side should win. I might need to introduce a version column.
If I had a last updated timestamp field I could use that. Or a version column but I am expressly trying to avoid introducing new columns to the schema. It means it's a lot harder.
"owner": "chronological"
"ownerlink": "/user/198/chronological"
"permalink": "/project/52001/?l=lt#t-146001"
"created": "2022-08-16T13:32:29.650418"
"happened": "2022-08-16T13:28:10"
"models":
- "id": "t-151001"
"url": "https://github.com/samsquire/postgres-sync"
"text": |
I implemented column hashing
"cost": -3.0
"currency": "HUR"
"assets": |
I implemented a hash that takes previous data and current column and the previous hash to produce a hash of the entire database.
This allows us to synchronize with the minimum of data transmissions when I write the synchronizer part which shall rehash all its own data, then retrieve the hash of a binary search of the sorted data.
"owner": "chronological"
"ownerlink": "/user/198/chronological"
"permalink": "/project/52001/?l=lt#t-151001"
"created": "2022-09-09T20:23:41.331046"
"happened": "2022-09-09T20:21:55"
"models":
"item:result:add": "/admin/hlog/task/add/?parent=52001"
"item:transfer:items":
"item:transfer:add": "/admin/hlog/operation/add/?parent=52001"
"item:comment:add": "/projects/addnote?parent=52001"
"item:comment:add:csrf_token": "2EuAIu4uKV9BPgs8Sm0EMrtRys5519Ltbr6OLFrntCL3qwyqntXdpmf55zh3N4yQ"
"item:comment:form": |
Pažymėkite, jei komentaras kelia naujų klausimų.
Pažymėkite, jei komentaras siųlo galimus sprendimus.
Pažymėkite, jei komentaras pateikia potencialiai naudingų faktų.
Pažymėkite, jei komentaras turi potencialiai darytinų darbų.