Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add SmallAutoField support #84

Open
timgraham opened this issue Nov 12, 2019 · 3 comments
Open

add SmallAutoField support #84

timgraham opened this issue Nov 12, 2019 · 3 comments

Comments

@timgraham
Copy link
Collaborator

Django 3.0 adds support for SmallAutoField, however, cockroachdb's unique_rowid() generates 8-byte values that are too large for int2 (2-byte). I'm not sure how to proceed.

Test failure:

======================================================================
ERROR: test_fk_to_smallautofield (many_to_one.tests.ManyToOneTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
psycopg2.errors.NumericValueOutOfRange: integer out of range for type int2 (column "country_id")


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/tim/code/django/tests/many_to_one/tests.py", line 583, in test_fk_to_smallautofield
    City.objects.create(country=us, name='Chicago')
  File "/home/tim/code/django/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/tim/code/django/django/db/models/query.py", line 433, in create
    obj.save(force_insert=True, using=self.db)
  File "/home/tim/code/django/django/db/models/base.py", line 746, in save
    force_update=force_update, update_fields=update_fields)
  File "/home/tim/code/django/django/db/models/base.py", line 784, in save_base
    force_update, using, update_fields,
  File "/home/tim/code/django/django/db/models/base.py", line 886, in _save_table
    results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw)
  File "/home/tim/code/django/django/db/models/base.py", line 925, in _do_insert
    using=using, raw=raw,
  File "/home/tim/code/django/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/tim/code/django/django/db/models/query.py", line 1204, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "/home/tim/code/django/django/db/models/sql/compiler.py", line 1344, in execute_sql
    cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/home/tim/code/django/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
  File "/home/tim/code/django/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/tim/code/django/django/db/backends/utils.py", line 86, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.DataError: integer out of range for type int2 (column "country_id")
@timgraham
Copy link
Collaborator Author

smallserial is used for SmallAutoField:

CREATE TABLE "smallauto_country" ("id" smallserial NOT NULL PRIMARY KEY, "name" varchar(50) NOT NULL);

However, it looks like cockroachdb uses unique_rowid() for that column. The 8-byte id is stored without error despite the fact that smallserial is 2-byte on PostgreSQL . The error comes when using the large id in a foreign key (which uses smallint):

CREATE TABLE "smallauto_city" ("id" bigserial NOT NULL PRIMARY KEY, "name" varchar(50) NOT NULL, "country_id" smallint NULL);

@rafiss
Copy link
Contributor

rafiss commented Nov 26, 2019

I wonder if we should change the smallserial alias in CockroachDB so it uses a different function that generates a smaller int? Like implement a unique_smallrowid() function, perhaps. I don't know if there are other implications with doing that.

FWIW, there also is a cluster setting called serial_normalization that allows the SERIAL type to use a sequence instead of unique_rowid(). I don't think we want to use it, but just listing it as an option. Details on the setting:
https://www.cockroachlabs.com/docs/releases/v2.1.0-beta.20180827.html
https://www.cockroachlabs.com/docs/stable/cluster-settings.html

cc @jordanlewis @apantel

@timgraham
Copy link
Collaborator Author

I'm guessing that unique_smallrowid() might be the way to go since CockroachDB's documentation says, "SERIAL is provided only for compatibility with PostgreSQL. New applications should use real data types and a suitable DEFAULT expression."

Django's AutoField and BigAutoField are both using integer data type with DEFAULT unique_rowid() rather than serial and bigserial.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants