文章/答案/技术大牛

发布

社区首页 >问答首页 >使用Android时，blob的最大大小是什么?这是在哪里记录的？

问使用Android时，blob的最大大小是什么?这是在哪里记录的？
EN

Stack Overflow用户

提问于 2022-08-28 17:42:40

回答 1查看 258关注 0票数 1

我使用Android并将ByteArray数据存储为BLOB。在一定大小以上，试图检索一行的尝试将失败，其中包含以下消息：

E/CursorWindow: The size of (0, 2) is too big (4194304), so replace the value to NULL

坠机的实际原因是：

    java.lang.NullPointerException: Parameter specified as non-null is null: method kotlin.jvm.internal.Intrinsics.checkNotNullParameter, parameter bytes

因此，在一定大小以上，CursorWindow无法加载blob，返回null，DAO方法抛出NPE，因为实体字段不可空。

我可以理解，在Sqlite/Room中会有一些关于blob大小的上限。在实验上，我确定我的特定设置(至少在某些情况下)是4194304 (4096*1024)。

我的问题是:这些文件在哪里？是否可以根据设备、房间/Sqlite版本等进行更改。我可以在运行时确定这个值吗？

我想在数据插入之前强制执行适当的字节数组限制，这样这个错误就不会发生，我正在寻找一种原则/可靠的方法来确定实际的限制。

更新

我认为@MikeT的答案是我们所能做的最好的。我们可以使用android.database.sqlite检索的最大行大小基本上是没有文档说明的；大多数将其设置为~2MB，但由于某种原因，我(有时)使用Android12将其钉住在~4MB。下面是@MikeT在Kotlin中的答案的一个版本，无论您的安装程序是对超大行抛出异常还是(像我的一样)对列返回一个空值(当使用包含非空字段的Room实体时，它将导致通过道方法抛出的NPE )。

    fun checkCursorWindowSize() {
        val context = this
        val TEST_TABLE_NAME = "tcw"
        val TEST_DATABASE_NAME = "tcw"
        val TEST_COLUMN_NAME = "testblob"
        val reducer = 1024 * 1024 / 16 /* size to reduce BLOB by per iteration */
        val allowance =
            1 /* size to reduce BLOB by for all extractions (reason for use probably best to not be on a power of 2 boundary????) */
        val ba = ByteArray(1024 * 1024 * 4) /* initial size of BLOB  when storing*/
        var csr: Cursor
        val tcwDbPath: String = context.getDatabasePath(TEST_DATABASE_NAME).getPath()
        val dbFile: File = context.getDatabasePath(TEST_DATABASE_NAME)
        val dbDirectory: File = dbFile.getParentFile()
        /* If the database exists then delete it = should never exist */if (dbFile.exists()) {
            dbFile.delete()
        }
        /* Ensure that the databases directory exists - otherwise openOrCreate fails */if (!dbDirectory.exists()) {
            Log.d(
                "DEBUGINFO",
                "Database directory " + dbDirectory.getPath()
                    .toString() + " does not exist - Creating"
            )
            dbDirectory.mkdirs()
        }

        /* Finally create the database */
        val db = SQLiteDatabase.openOrCreateDatabase(tcwDbPath, null, null)
        /* Do everything inside a transaction to reduce writes */db.beginTransaction()
        /* Create the table into which the BLOB will be loaded */db.execSQL("CREATE TABLE IF NOT EXISTS $TEST_TABLE_NAME ($TEST_COLUMN_NAME BLOB)")
        /* Insert the over-sized BLOB */
        val cv = ContentValues()
        cv.put(TEST_COLUMN_NAME, ba)
        db.insert(TEST_TABLE_NAME, null, cv)
        /* Prepare to repeatedly extract an eve decreasing part of the blob */
        var retrieveSize = ba.size /* first try to retrieve entire BLOB */
        var caught = 0 /* the number of caught exceptions */
        /* Try 1024 attempts at getting smaller BLOB */
        for (i in 0..1024) {
            Log.d("DEBUG", "Iteration is " + (i + 1) + ". Retrieve Size is " + retrieveSize)
            try {
                /* Build the query to extract part of the blob (using the SQLite substr function) according to
                    the current retrieve size less the allowance
                    only getting 1 row (should only be 1 row)
                    NOTE!! retrieving an oversize BLOB does not fail here, rather the failure is when attempting to read the data (getBlob method)
                 */
                csr = db.query(
                    TEST_TABLE_NAME,
                    arrayOf("substr(" + TEST_COLUMN_NAME + ",1," + (retrieveSize - allowance) + ") AS " + TEST_COLUMN_NAME),
                    null,
                    null,
                    null,
                    null,
                    null,
                    "1"
                )
                var test: ByteArray?=null
                while (csr.moveToNext()) {
                    test = csr.getBlob(0) /* get the Blob from the  1st (only) column */
                    Log.d("DEBUG", "Received value: ${test?.size}")
                }
                if (test != null) {
                    break
                }

            } catch (e: SQLException) {
                caught++
                //e.printStackTrace();
            } finally {
                retrieveSize =
                    retrieveSize - reducer /* reduce the retrieveSize ready for the next iteration (if one )*/
                if (retrieveSize < 0) break
            }
        }
        Log.d("DEBUGINFO", "Caught $caught SQlite Exceptions")
        /* Done with the database so clean up */db.endTransaction()
        db.close()
        File(tcwDbPath).delete()
        /* adjust the retrieve size as it has been adjusted in preparation for the next iteration */
        Log.d("DEBUGINFO","MaxCursorSize = " + (retrieveSize + reducer).toLong())
    }

android-room

android-sqlite

回答 1

Stack Overflow用户

回答已采纳

发布于 2022-08-28 19:33:38

您的问题不是BLOB的最大大小，而是游标窗口的最大大小，它是特定于Android API/实现的，限制范围从1Mb到4Mb不等(如果我没记错的话)，这取决于安卓的版本(后面的版本具有更大的大小)。

游标窗口包装在游标实现中，对于Room，游标实现封装在方便的方法中，因此确定大小(虽然可能是可能的)可能是一项困难/复杂的任务。

可以绕过这个问题，即游标窗口必须能够存储整行(因此其他列会影响Blob本身的大小)。您可以提取块的部分(块)，然后将它们组合起来。使用小于1Mb的块(可能要少得多)将覆盖所有的限制。

然而，对于如此大量的数据，建议将数据存储为文件，然后存储足够的文件路径，以便能够检索该文件。

这里有两个答案(但不使用Room )，它们从大于游标窗口限制的Blob中检索数据。

，你知道我如何确定一个特定版本的Android的限制大小(或者在运行时确定我的应用程序所使用的版本)吗？我在用于Android12的AOSP代码中发现了config_cursorWindowSize = 2048 (这是我目前正在测试的)，但这不符合这样一个事实，即我已经有了高达4MB/ blob的查询(实际上，每一行都是这样)。

从有限的测试来看，(API 28和31)是一种方式，但我怀疑这可能会改变(两者都显示2MB，即2097152)，这取决于其他用途，因为CursorWindow可以使用传递windowSize的构造函数来构造。

下面是基本/有限的java方法

exception

finally

创建一个数据库和一个表，其中只有1列将一个超大的BLOB (4 4Mb)作为唯一行插入，然后

将逐步尝试提取BLOB的一个较小的部分(最初都是这样)，直到可以在不使用

的情况下提取数据，从而清除数据库

。

long checkCursorWindowSize(Context context) {
        final String TEST_TABLE_NAME = "tcw";
        final String TEST_DATABASE_NAME = "tcw";
        final String TEST_COLUMN_NAME = "testblob";
        final int reducer = 1024 * 1024 / 16; /* size to reduce BLOB by per iteration */
        final int allowance = 1; /* size to reduce BLOB by for all extractions (reason for use probably best to not be on a power of 2 boundary????) */
        byte[] ba = new byte[1024 * 1024 * 4]; /* initial size of BLOB  when storing*/

        Cursor csr;
        String tcwDbPath = context.getDatabasePath(TEST_DATABASE_NAME).getPath();
        File dbFile = context.getDatabasePath(TEST_DATABASE_NAME);
        File dbDirectory = dbFile.getParentFile();
        /* If the database exists then delete it = should never exist */
        if (dbFile.exists()) {
            dbFile.delete();
        }
        /* Ensure that the databases directory exists - otherwise openOrCreate fails */
        if (!dbDirectory.exists()) {
            Log.d("DEBUGINFO", "Database directory " + dbDirectory.getPath() + " does not exist - Creating");
            dbDirectory.mkdirs();
        }

        /* Finally create the database */
        SQLiteDatabase db = SQLiteDatabase.openOrCreateDatabase(tcwDbPath,null,null);
        /* Do everything inside a transaction to reduce writes */
        db.beginTransaction();
        /* Create the table into which the BLOB will be loaded */
        db.execSQL("CREATE TABLE IF NOT EXISTS " + TEST_TABLE_NAME + " (" + TEST_COLUMN_NAME + " BLOB)");
        /* Insert the over-sized BLOB */
        ContentValues cv = new ContentValues();
        cv.put(TEST_COLUMN_NAME,ba);
        db.insert(TEST_TABLE_NAME,null,cv);
         /* Prepare to repeatedly extract an eve decreasing part of the blob */
        int retrieveSize = ba.length; /* first try to retrieve entire BLOB */
        int caught = 0; /* the number of caught exceptions */
        /* Try 1024 attempts at getting smaller BLOB */
        for (int i=0; i <= 1024; i++  ) {
            Log.d("DEBUG","Iteration is " + (i + 1) + ". Retrieve Size is " + retrieveSize);
            try {
                /* Build the query to extract part of the blob (using the SQLite substr function) according to
                    the current retrieve size less the allowance
                    only getting 1 row (should only be 1 row)
                    NOTE!! retrieving an oversize BLOB does not fail here, rather the failure is when attempting to read the data (getBlob method)
                 */
                csr = db.query(TEST_TABLE_NAME, new String[]{"substr(" + TEST_COLUMN_NAME + ",1," + (retrieveSize - allowance) + ") AS " + TEST_COLUMN_NAME},null,null,null,null,null,"1");
                while (csr.moveToNext()) {
                    byte[] test = csr.getBlob(0); /* get the Blob from the  1st (only) column */
                }
                break; /* if no Exception  then all done */
            } catch (SQLException e) {
                retrieveSize = retrieveSize - reducer; /* reduce the retrieveSize ready for the next iteration (if one )*/
                caught++;
                //e.printStackTrace();
            }
        }
        Log.d("DEBUGINFO","Caught " + caught + " SQlite Exceptions");
        /* Done with the database so clean up */
        db.endTransaction();
        db.close();
        new File(tcwDbPath).delete();
        /* adjust the retrieve size as it has been adjusted in preparation for the next iteration */
            return retrieveSize + reducer;
    }

有限测试在MainActivity中使用了以下内容：

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    Log.d("DEBUGINFO","MaxCursorSize = " + checkCursorWindowSize(this));

}

在API 28 :- D/DEBUGINFO: MaxCursorSize = 2097152上，整个日志是：-

2022-08-29 10:58:06.798 D/DEBUG: Iteration is 16. Retrieve Size is 3211264
2022-08-29 10:58:06.802 W/CursorWindow: Window is full: requested allocation 3211263 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.802 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3211263) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.802 D/DEBUG: Iteration is 17. Retrieve Size is 3145728
2022-08-29 10:58:06.806 W/CursorWindow: Window is full: requested allocation 3145727 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.807 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3145727) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.807 D/DEBUG: Iteration is 18. Retrieve Size is 3080192
2022-08-29 10:58:06.810 W/CursorWindow: Window is full: requested allocation 3080191 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.810 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3080191) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.810 D/DEBUG: Iteration is 19. Retrieve Size is 3014656
2022-08-29 10:58:06.813 W/CursorWindow: Window is full: requested allocation 3014655 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.813 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,3014655) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.813 D/DEBUG: Iteration is 20. Retrieve Size is 2949120
2022-08-29 10:58:06.818 W/CursorWindow: Window is full: requested allocation 2949119 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.819 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2949119) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.819 D/DEBUG: Iteration is 21. Retrieve Size is 2883584
2022-08-29 10:58:06.824 W/CursorWindow: Window is full: requested allocation 2883583 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.824 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2883583) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.825 D/DEBUG: Iteration is 22. Retrieve Size is 2818048
2022-08-29 10:58:06.829 W/CursorWindow: Window is full: requested allocation 2818047 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.829 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2818047) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.829 D/DEBUG: Iteration is 23. Retrieve Size is 2752512
2022-08-29 10:58:06.832 W/CursorWindow: Window is full: requested allocation 2752511 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.832 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2752511) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.832 D/DEBUG: Iteration is 24. Retrieve Size is 2686976
2022-08-29 10:58:06.837 W/CursorWindow: Window is full: requested allocation 2686975 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.838 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2686975) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.838 D/DEBUG: Iteration is 25. Retrieve Size is 2621440
2022-08-29 10:58:06.842 W/CursorWindow: Window is full: requested allocation 2621439 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.842 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2621439) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.842 D/DEBUG: Iteration is 26. Retrieve Size is 2555904
2022-08-29 10:58:06.846 W/CursorWindow: Window is full: requested allocation 2555903 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.846 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2555903) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.846 D/DEBUG: Iteration is 27. Retrieve Size is 2490368
2022-08-29 10:58:06.849 W/CursorWindow: Window is full: requested allocation 2490367 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.849 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2490367) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.849 D/DEBUG: Iteration is 28. Retrieve Size is 2424832
2022-08-29 10:58:06.853 W/CursorWindow: Window is full: requested allocation 2424831 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.853 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2424831) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.854 D/DEBUG: Iteration is 29. Retrieve Size is 2359296
2022-08-29 10:58:06.857 W/CursorWindow: Window is full: requested allocation 2359295 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.858 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2359295) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.858 D/DEBUG: Iteration is 30. Retrieve Size is 2293760
2022-08-29 10:58:06.862 W/CursorWindow: Window is full: requested allocation 2293759 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.862 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2293759) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.862 D/DEBUG: Iteration is 31. Retrieve Size is 2228224
2022-08-29 10:58:06.865 W/CursorWindow: Window is full: requested allocation 2228223 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.865 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2228223) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.865 D/DEBUG: Iteration is 32. Retrieve Size is 2162688
2022-08-29 10:58:06.870 W/CursorWindow: Window is full: requested allocation 2162687 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.870 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2162687) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.870 D/DEBUG: Iteration is 33. Retrieve Size is 2097152
2022-08-29 10:58:06.874 W/CursorWindow: Window is full: requested allocation 2097151 bytes, free space 2096720 bytes, window size 2097152 bytes
2022-08-29 10:58:06.875 E/SQLiteQuery: exception: Row too big to fit into CursorWindow requiredPos=0, totalRows=1; query: SELECT substr(testblob,1,2097151) AS testblob FROM tcw LIMIT 1
2022-08-29 10:58:06.875 D/DEBUG: Iteration is 34. Retrieve Size is 2031616
2022-08-29 10:58:06.881 D/DEBUGINFO: Caught 33 SQlite Exceptions
2022-08-29 10:58:06.904 D/DEBUGINFO: MaxCursorSize = 2097152

在API 31上，结果完全相同，时间/时间(即MaxCursorSize )被计算为2097512。

但是，例如，添加另一个列，则必须将其他数据加载到CursorWindow中，结果并不能真正反映Blob的大小。

因此，如果任何依赖于一个

所能容纳的极限，那么它很可能只会导致一些问题。

票数 2

页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持

原文链接：

https://stackoverflow.com/questions/73520880

复制

相似问题

问使用Android时，blob的最大大小是什么?这是在哪里记录的？
EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用Android时，blob的最大大小是什么?这是在哪里记录的？EN

回答 1

Stack Overflow用户

社区

活动

圈层

关于

腾讯云开发者

热门产品

热门推荐

更多推荐

问使用Android时，blob的最大大小是什么?这是在哪里记录的？
EN