Feature #2629 : Ability to specify length for index segments to be indexed on char-based columns

[hvlad]

With this feature it is possible to create index on long text columns specifying max count of leading characters of string that is included into index key.

The syntax for index segment definition: <column> [(<length>)], where column is table column name and length is optional maximum count of characters included into index key.

The length could be set for CHAR | VARCHAR | BINARY | VARBINARY types, blobs is not supported currently.

ODS changes:

1. The new column RDB$INDEX_SEGMENTS.RDB$CHARACTER_LENGTH
2. The new field irtd_length in index key descriptor, see struct Ods::irtd

[hvlad]

The questions for further consideration:

1. should we support BLOB's indexing ?
2. UNIQUE indices looks possible but with some warnings:

• there could be a lot of duplicate keys for different strings with the same prefix, it makes unique check to take longer,
• currently CREATE UNIQUE INDEX report false duplicate errors as they produced when sorting and only keys are compared, not actual values from records,
• at the same time CREATE UNIQUE INDEX ... CONCURRENTLY successfully create and validate index, as its check for duplicates uses values from table records,
• thus the question - should we change duplicates check used by traditional CREATE UNIQUE INDEX or disable feature for UNIQUE indices ?

[sim1984]

The questions for further consideration:

1. should we support BLOB's indexing ?

I think this is a dubious feature. How many people search BLOBs using blob_field STARTING WITH ?
A partial comparison using blob_field = ? is also questionable. If the BLOB itself is large, it won't be effective anyway.

BLOBs require slightly different indexes, such as inverted ones for FTS and the like.

2. UNIQUE indices looks possible but with some warnings:

• there could be a lot of duplicate keys for different strings with the same prefix, it makes unique check to take longer,
• currently CREATE UNIQUE INDEX report false duplicate errors as they produced when sorting and only keys are compared, not actual values from records,
• at the same time CREATE UNIQUE INDEX ... CONCURRENTLY successfully create and validate index, as its check for duplicates uses values from table records,
• thus the question - should we change duplicates check used by traditional CREATE UNIQUE INDEX or disable feature for UNIQUE indices ?

Creating a unique index based on part of a key makes rather strange logical sense. CREATE UNIQUE INDEX ... CONCURRENTLY, while it performs uniqueness validation differently, still creates a truly unique index, albeit with temporarily violated uniqueness. This can only confuse the user. Ultimately, this feature is primarily needed to speed up searches on long character columns, not to create uniqueness constraints.

[hvlad]

Not much opinions... ok.

I agree that there is almost no reasons to support indexing of BLOB's prefix.

As for UNIQUE indices, the only reason I heard that sounds reasonable: when some relatively long column uses national charset (single-byte) and was converted into UTF8 (after upgrade) - and now can't be indexed due to key size limit.

[dyemanov]

1. No need to index blob prefixes
2. I'd vote for supporting UNIQUE prefix-based indices, even if the duplicate checking is slower (provided that regular UNIQUE indices are not affected).

[hvlad]

Ok.

I'm going to change regular CREATE INDEX (for unique prefix-based indices only) to run duplicates check after build of b-tree - like concurrent index creation do.