database - Is there any transparent deduplication for large binary data within PostgreSQL? -
we have app stores large binary data large objects within postgresql database , have use case operate on data in such way know parts of saved data saved once again in different combination. going split files , combine them in different ways, want preserve original files additionally new combined ones.
this sounds use case consider deduplication. aware of works directly , transparent within postgres, such don't need re-invent wheel? storage layer or plugin add postgres handles deduplication on it's own @ least whole database? or maybe library works wrapper around large object function of postgres our app use , wrapper lib duplication part, maybe adds tables bookkeeping , stuff?
we aware of filesystems support deduplication , 1 possibility use backend postgres' data storage. option save new data outside of postgres within deduplicating filesystem. prefer within postgres can dumped , backed transactions etc.
thanks hints!
no, there not (as of 9.4, anyway).
postgresql compresses each individual item, doesn't cross-item compression or deduplication.
the toast mechanism means postgresql should able deduplicate values in toastable types effort, it's matter of implementing it.
patches welcome ;-)
Comments
Post a Comment