Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Zip multiple exported QIF files into one for all export destinations #714

Merged
merged 3 commits into from
Aug 2, 2017
Merged

Zip multiple exported QIF files into one for all export destinations #714

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c4f6a26
Always Zip files when exporting to QIF
rivaldi8 Aug 1, 2017
1711414
Don't zip exported files when exporting to a URI (SAF)
rivaldi8 Aug 1, 2017
fad4aaa
Avoid creating an empty ZIP file when exporting to QIF
rivaldi8 Aug 2, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 3 additions & 18 deletions app/src/main/java/org/gnucash/android/export/ExportAsyncTask.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -283,25 +283,10 @@ private void moveExportToUri() throws Exporter.ExporterException {
if (mExportedFiles.size() > 0){
try {
OutputStream outputStream = mContext.getContentResolver().openOutputStream(exportUri);
ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rivaldi8 While going through the commits to create a CHANGELOG for the next release, I revisited this issue.
I think we should zip the file which we export (even if it is just a single one), because it saves bandwidth for user sync. Also, some users have really large databases which lead to large XML outputs (always exports the whole database).

One way to remedy this would be to make the GncXmlExporter always exports zipped files (without the .zip extension, only .gnca - .gnca will then be considered a zipped format). Then we do not need to worry about it here (or anywhere else).

GnuCash desktop can handle zipped XML files for export, as can our app. It will not be directly user-readable, but then again, XML is not really meant to be user-readable. And those who want to read it can just unzip it once.

What do you think?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@codinguser Yes, I agree with you, but I think you really mean gzipped (GZIPOutputStream, like here) instead of zipped (ZipOutputStream). That's the compression algorithm used in .gnca files, isn't it?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rivaldi8 yes, I meant gzipped, sorry :)

byte[] buffer = new byte[1024];
for (String exportedFile : mExportedFiles) {
File file = new File(exportedFile);
FileInputStream fileInputStream = new FileInputStream(file);
zipOutputStream.putNextEntry(new ZipEntry(file.getName()));

int length;
while ((length = fileInputStream.read(buffer)) > 0) {
zipOutputStream.write(buffer, 0, length);
}
zipOutputStream.closeEntry();
fileInputStream.close();
}
zipOutputStream.close();
// Now we always get just one file exported (QIFs are zipped)
moveFile(mExportedFiles.get(0), outputStream);
} catch (IOException ex) {
Log.e(TAG, "Error when zipping QIF files for export");
ex.printStackTrace();
Crashlytics.logException(ex);
throw new Exporter.ExporterException(mExportParams, "Error when moving file to URI");
}
}
}
Expand Down
18 changes: 17 additions & 1 deletion app/src/main/java/org/gnucash/android/export/qif/QifExporter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,14 @@
import android.content.ContentValues;
import android.database.Cursor;
import android.database.sqlite.SQLiteDatabase;
import android.support.annotation.NonNull;

import org.gnucash.android.db.adapter.AccountsDbAdapter;
import org.gnucash.android.db.adapter.TransactionsDbAdapter;
import org.gnucash.android.export.ExportParams;
import org.gnucash.android.export.Exporter;
import org.gnucash.android.model.Commodity;
import org.gnucash.android.util.FileUtils;
import org.gnucash.android.util.PreferencesHelper;
import org.gnucash.android.util.TimestampHelper;

Expand All @@ -38,6 +40,7 @@
import java.io.OutputStreamWriter;
import java.math.BigDecimal;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Locale;

Expand All @@ -52,6 +55,8 @@
* @author Yongxin Wang <fefe.wyx@gmail.com>
*/
public class QifExporter extends Exporter{
private static final String TAG = "QifExporter";

/**
* Initialize the exporter
* @param params Export options
Expand Down Expand Up @@ -232,12 +237,23 @@ public List<String> generateExport() throws ExporterException {

/// export successful
PreferencesHelper.setLastExportTime(TimestampHelper.getTimestampFromNow());
return splitQIF(file);
List<String> exportedFiles = splitQIF(file);
if (exportedFiles.isEmpty())
return Collections.emptyList();
else
return zipQifs(exportedFiles);
} catch (IOException e) {
throw new ExporterException(mExportParams, e);
}
}

@NonNull
private List<String> zipQifs(List<String> exportedFiles) throws IOException {
String zipFileName = getExportCacheFilePath() + ".zip";
FileUtils.zipFiles(exportedFiles, zipFileName);
return Collections.singletonList(zipFileName);
}

/**
* Splits a Qif file into several ones for each currency.
*
Expand Down
34 changes: 34 additions & 0 deletions app/src/main/java/org/gnucash/android/util/FileUtils.java
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package org.gnucash.android.util;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.util.List;
import java.util.zip.ZipEntry;
import java.util.zip.ZipOutputStream;

/**
* Misc methods for dealing with files.
*/
public final class FileUtils {
public static void zipFiles(List<String> files, String zipFileName) throws IOException {
OutputStream outputStream = new FileOutputStream(zipFileName);
ZipOutputStream zipOutputStream = new ZipOutputStream(outputStream);
byte[] buffer = new byte[1024];
for (String fileName : files) {
File file = new File(fileName);
FileInputStream fileInputStream = new FileInputStream(file);
zipOutputStream.putNextEntry(new ZipEntry(file.getName()));

int length;
while ((length = fileInputStream.read(buffer)) > 0) {
zipOutputStream.write(buffer, 0, length);
}
zipOutputStream.closeEntry();
fileInputStream.close();
}
zipOutputStream.close();
}
}