Remove "<br/> Empty media" from Telegram messages

[aberenguel]

I'm using IPED 4.0.6.

I processed an UFDR file and some Telegram messages the user sent have the term "< br/> Empty media" in the trailing.

[aberenguel]

As workaround, how can I edit the HTML file in the case? I haven't found it.

[lfcnassif]

As workaround, how can I edit the HTML file in the case? I haven't found it.

Well, it is not that straightforward, you would need to take the item hash and go to %case%/iped/stotage/storage-X.db where X is the decimal number correspondent to the first hex char of the hash value. Then open that SQLITE DB, go to t1 table and find the item by id (it's equal to the hash value). The item content is into the data column, but it is compressed using org.apache.commons.compress.compressors.gzip.GzipCompressorOutputStream

[lfcnassif]

I'm tagging this as bug because the <br/> tag shouldn't be printed. Possibly it started to be printed when we started to HTML encode all message texts to avoid javascript vulnerabilities. The triggering line is in telegram-decoder dependency:
https://github.com/sepinf-inc/telegram-decoder/blob/35d89dfb42a81de8c6a669dd9227bb64ae27276c/telegram-decoder-impl/src/main/java/telegramdecoder/DecoderTelegram.java#L206

All <br/> should be removed from that class. But seems to me the "Empty media" string is intentional. @hauck-jvsh should we keep (and possibly translate) the "Empty media" string or could it be removed?

[aberenguel]

To whom is interested in the fix after the case is processed:

package general;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.File;
import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.charset.StandardCharsets;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.zip.GZIPOutputStream;

import org.apache.commons.compress.compressors.gzip.GzipCompressorInputStream;
import org.apache.commons.io.FileUtils;

public class FixTelegram {

    private static String[] hashes;

    public static void main(String[] args) throws IOException {

        hashes = FileUtils.readFileToString(new File("case/chats-telegram-hashes.txt"),
                Charset.defaultCharset()).split("\n");

        for (int i = 0; i < 16; i++) {

            fixDatabase(i);
        }

    }

    private static void fixDatabase(int index) throws IOException {

        File file = new File("case/IPED/iped/storage/storage-" + index + ".db");

        try (Connection connection = DriverManager.getConnection("jdbc:sqlite:" + file.getAbsolutePath())) {

            System.out.println("Conexão realizada em " + file);

            PreparedStatement stmt = connection.prepareStatement("select data from t1 where id = ?");

            PreparedStatement stmtUpdate = connection.prepareStatement("update t1 set data = ? where id = ?");

            for (String hash : hashes) {

                if (hash.startsWith(Integer.toHexString(index).toUpperCase())) {

                    stmt.setString(1, hash);
                    ResultSet resultSet = stmt.executeQuery();

                    byte[] compressedBytes = resultSet.getBytes(1);

                    GzipCompressorInputStream gzipInput = new GzipCompressorInputStream(
                            new ByteArrayInputStream(compressedBytes));

                    String html = new String(gzipInput.readAllBytes(), StandardCharsets.UTF_8);

                    String newHtml = html.replace("&lt;br/&gt; Empty media", "");

                    if (newHtml.equals(html)) {
                        continue;
                    }
                    
                    ByteArrayOutputStream output = new ByteArrayOutputStream();
                    try (GZIPOutputStream gzipOutput = new GZIPOutputStream(output)) {
                        gzipOutput.write(newHtml.getBytes(StandardCharsets.UTF_8));
                    }

                    stmtUpdate.setBytes(1, output.toByteArray());
                    stmtUpdate.setString(2, hash);

                    stmtUpdate.executeUpdate();
                }
            }
        } catch (SQLException e) {
            System.out.println(e.getMessage());
        }
    }
}

[lfcnassif]

Thanks @aberenguel. How do those messages render in Telegram native application? Is there any kind of "empty media" indication, like an empty picture? Or just the text before <br/> Empty media?

[lfcnassif]

Or how Cellebrite software render those messages?

[lfcnassif]

@aberenguel said original evidence is not available anymore to check, and in Cellebrite software those messages are shown just with the text without any media indication, so let's make the behavior the same.

[lfcnassif]

This commit tries to fix the issue:
sepinf-inc/telegram-decoder@aa38ab8

I'm just in doubt about the old line 175, if it should be kept or not since it's not the issue reported here, but I commented it out for now (it looks similar). @hauck-jvsh please let me know if that is not ok.

[lfcnassif]

I'm just in doubt about the old line 175, if it should be kept or not since it's not the issue reported here, but I commented it out for now (it looks similar). @hauck-jvsh please let me know if that is not ok.

I just found one single occurrence of it in a case and removing the "Empty media" string would leave an empty message balloon, so I'll revert that specific change.