Fix dbrestore of sqlite database to like newlines

[rgaiacs]

Related to #238.

On Textfields that have line breaks, sqlite reports "unrecognized token"
because it doesn't keep the previous line in memory to be ammended by
the current line.

This pull requests implements a very simple logic to only send SQL
INSERT commands when the line finished with ");".

Closes #254

[ZuluPro]

Hi @rgaiacs
Very great work, could you just add unit test to ensure your PR ?

[rgaiacs]

Hi @ZuluPro, could you help me with the unit test? I'm not sure if dbbackup/tests/commands/test_dbrestore.py is the file that I need to edit.

[ZuluPro]

Hello @rgaiacs
There are the following things to do:

• Create a model for TextField in dbbackup.tests.testapp.models.TextModel
• Add a specific test using TextField in dbbackup.tests.test_connectors. SqliteConnectorTest. test_restore_dump_with_textarea_and_newline

Thank you very much !

[ZuluPro]

Hello @rgaiacs
Any news ?

[rgaiacs]

Any news ?

Still trying to write the tests. Unit tests isn't my strongest skill. :-(

[rgaiacs]

@ZuluPro I try to do as instructed but I'm receiving

sqlite3.OperationalError: no such table: testapp_textmodel
~~~

[ZuluPro]

@rgaiacs
Thanks for you work.
Did you do the makemigrations ?

[rgaiacs]

Did you do the makemigrations ?

I didn't. I executed makemigrations and get some UserWarning when running the unittests. Can you have a look and provide feedback. Thanks very much for your help.

[ZuluPro]

@rgaiacs
What is your UserWarning ? Is it a SQLite one ?
Could you just test the integrity of the data, if after restore "foo\nbar" is the same ?

[ZuluPro]

@rgaiacs Any news ?

[Lokkook]

Hi !
@ZuluPro, @jonathan-s : Is this merge request planned? Seems pretty useful 👍
thanks!

[dtasev]

@Lokkook I wouldn't rely on it, it seems #258 has been long abandoned despite the work being there and just needing a test. Unfortunate, it seems like a great tool but if it doesn't work for the simplest case of SQLite, it's not going anywhere near production

[ZuluPro]

@rgaiacs @Lokkook @dtasev
The PR isn't in mergeable state.

[dtasev]

I agree, this PR is quite old so it's likely the project has moved quite a bit forward. It just seemed a shame as, at least from the comments, it sounds like it was somewhat close to being done in 2017. If people happen to come across this issue, this was my solution for SQLite in the end:

To dump the db & encrypt it:

manage.py dumpdata --natural-foreign --natural-primary -e contenttypes -e auth.Permission > dbdump.json
gpg --passphrase $PASSPHRASE --quiet --batch --yes -c dbdump.json && rm dbdump.json

To restore:

gpg --passphrase $PASSPHRASE --quiet --batch --yes dbdump.json.gpg
manage.py loaddata dbdump.json

[demonhunter3333]

it's crazy that this package doesn't even work with SQLite😢

[Archmonger]

Hey @demonhunter3333

I took over this project not too long ago. I'm aware of several issues with the current design, and am proposing an overhaul to fix them. See the following for more info:

• Deciding the Project's Future #423

[Archmonger]

Closing in favor of

• Fix dbrestore on SQLite #383