Android: Position in Collection not supported by Flatlist, SectionList, VirtualizedList, or ScrollView

[fabOnReact]

Issue facebook/react-native#30977
Pull Request facebook/react-native#33180
Related facebook/react-native#31666

[fabOnReact]

<FlatList numColumns={undefined} /> should not trigger Runtime Error NoSuchKey exception columnCount when enabling TalkBack.

<FlatList numColumns={undefined} />

Expected Result:
No Runtime Error

Actual Result:
Enabling TalkBack triggers a Runtime Error NoSuchKeyException columnCount

CLICK TO OPEN RUNTIME ERROR

CLICK TO OPEN DETAILS

https://github.com/fabriziobertoglio1987/react-native/blob/e36f46a003a0c279d6d7e84bdfefc0b1a61d5cb7/Libraries/Lists/FlatList.js#L671-L675

https://github.com/fabriziobertoglio1987/react-native/blob/e36f46a003a0c279d6d7e84bdfefc0b1a61d5cb7/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactHorizontalScrollView.java#L201

numColums default was removed with commit 7d5895d

CLICK TO OPEN TESTS RESULTS - WITHOUT RUNTIME

numColumnsNoRuntime.mp4

CLICK TO OPEN TESTS RESULTS - WITH RUNTIME

numColumnsRuntime.mp4

Solved with fabOnReact/react-native@062cdcd

[fabOnReact]

Scenarios Previously Tested in facebook/react-native#31666 (comment) (VIDEO1 and VIDEO2)

• Horizontal FlatList
• Vertical FlatList
• Horizontal FlatList
• Horizontal FlatList when fixed layout
• FlatList when numColumns > 1
• FlatList when numColumns > 1 and fixed layout
• Simple 1 column FlatList
• Simple 1 column FlatList - Fixed layout
• Nested FlatList

New Scenarios

• read diff Android: Position in Collection not supported by Flatlist, SectionList, VirtualizedList, or ScrollView #6 (comment)
  * understand the use of CollectionInfo properties by changing the values of rowSpan, columnSpan and columnIndex
• change values of different props used in the branch (for ex. numColumns to undefined)
• find potential issues in diff
• do random tests
• learn more about TalkBack with FlatList.
  - Special TalkBack features with Android ScrollView
  - TalkBack Shortcuts
  - Change TalkBack settings and test them with FlatList

[fabOnReact]

Columns in FlatList with multiple columns are not announced

• A blind user uses TalkBack to navigate items in the FlatList.
• The FlatList has 2 columns and 6 rows (grid layout)
• TalkBack announces each of the items and their column and row number

Expected Result:
Talkback announces:

1. Marketplace row 1 column 1
2. FriendsList row 1 column 2
3. Groups row 2 column 1
4. Pages row 2 column 2
5. Fifth Item row 3 column 1
6. Sixth Item row 3 column 2

Actual Result:
Talkback announces:

1. Marketplace row 1
2. FriendsList
3. Groups row 2
4. Pages
5. Fifth Item row 3
6. Sixth Item

The TalkBack announcements are correct when using 1 column.

CLICK TO OPEN EXAMPLE

data

const DATA = [
  {
    id: 'bd7acbea-c1b1-46c2-aed5-3ad53abb28ba',
    title: 'Marketplace',
  },
  {
    id: '3ac68afc-c605-48d3-a4f8-fbd91aa97f63',
    title: 'FriendsList',
  },
  {
    id: '58694a0f-3da1-471f-bd96-145571e29d72',
    title: 'Groups',
  },
  {
    id: 'bd7acbea-c1b1-46c2-aed5-3ad53abb8bbb',
    title: 'Pages',
  },
  {
    id: '3ac68afc-c605-48d3-a4f8-fbd91aa97676',
    title: 'Fifth Item',
  },
  {
    id: '58694a0f-3da1-471f-bd96-145571e27234',
    title: 'Sixth Item',
  },
];
// and more items

JSX

<>
      <FlatList
        keyExtractor={item => item.id}
        numColumns={2}
        pagingEnabled={true}
        data={DATA}
        renderItem={renderItem}
        keyExtractor={item => item.toString()}
      />
</>

CLICK TO OPEN TESTS RESULTS - EXPECTED BEHAVIOUR

gridViewExpectedBehaviour.mp4

CLICK TO OPEN TESTS RESULTS - ACTUAL BEHAVIOUR

columnsInSpanNotAnnounced.mp4

[fabOnReact]

SOLUTION FOR ISSUE #6 (comment)

The issue is caused by this 2 lines. Commenting them and replacing them with a random integer number fixes the issue.

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L210-L211

• The correct column numbers are announced when moving between different columns.
• The correct row numbers are announced when moving between different rows.

CLICK TO OPEN VIDEO - NOT ANNOUNCING COLUMN

notAnnouncingColumn.mp4

CLICK TO OPEN VIDEO - ANNOUNCING COLUMN

announcingColumn.mp4

CLICK TO OPEN VIDEO - ANNOUNCING COLUMN AND ROW

announcingColumnAndRow.mp4

CLICK TO OPEN VIDEO - SCROLL PAGE BEFORE

With 3 columns - Before the Change

verifyBehaviourWithoutChange.mp4

CLICK TO OPEN VIDEO - SCROLL PAGE AFTER

With 3 columns - After the Change

notAnnouncingCorrectListCount.mp4

CLICK TO OPEN MORE INFO

Removing the lines below (or replacing grid with list) would remove the row/column announcement.

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L409

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L698

more info at #6 (comment)

The index is set correctly for columns 1 and 2, but TalkBack does not announce them.

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L620

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L237

The reason is that our Java Implementation of FlatList numColums does not use GridView, but simply uses ScrollView formatting data as a 2 columns table.

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/Libraries/Lists/FlatList.js#L492-L506

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/views/scroll/ReactScrollView.java#L67

https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/ScrollView.java#L56-L80

https://github.com/aosp-mirror/platform_frameworks_base/blob/1ac46f932ef88a8f96d652580d8105e361ffc842/core/java/android/widget/GridView.java#L54-L69

https://github.com/fabriziobertoglio1987/react-native-notes/blob/875076952dc8658d2cd61026411dbc24e05e90a3/ReactAndroid/src/main/java/com/facebook/react/uimanager/ReactAccessibilityDelegate.java#L146

Removing the above line:

• TalkBack announces the item when scrolling pages
• Rows are not announced anymore

Adding the accessibilityRole grid adds the announcement for the rows.

CLICK TO OPEN TESTS RESULTS

removingRole.mp4

[fabOnReact]

Using height instead of width

Solves #6 (comment) #6 (comment).
The method from ReactHorizontalScrollView scrolls in a different direction.
computeScrollDeltaToGetChildRectOnScreen is inherited from ScrollView. Video of the tests in #6 (comment).

  private boolean isPartiallyScrolledInView(View descendent) {
    int scrollDelta = getScrollDelta(descendent);
    descendent.getDrawingRect(mTempRect);
    return scrollDelta != 0 && Math.abs(scrollDelta) < mTempRect.height();
  }

[fabOnReact]

Not announcing correctly the last page (partial scroll)

Solves #6 (comment) #6 (comment).
The issue was caused by ScrollDelta with value 0 (solved with fabOnReact/react-native@ee23f45).

2022-02-16.12-51-16.mp4

[fabOnReact]

TalkBack announces pages when scrolling down a GridView

TalkBack announces showing item 10 to 21 of 29 when scrolling pages in a GridView.

fabOnReact/MyApplication@0bfa67e

2022-02-17.10-52-22.mp4

[fabOnReact]

TalkBack announces pages and cells with Horizontal Flatlist in the Paper Renderer

Expected/Actual Result:
Talkback announces the selected item row and column changes:

1. 1 Item one of 29
2. 2 Item two of 29

Expected/Actual Result:
Talkback announces the pages when scrolling down:

Thirteen Item row 5 column 1 then says showing item 2 to 4 of 29

CLICK TO OPEN SOURCECODE

https://github.com/fabriziobertoglio1987/ReactNativeAwesomeProject/blob/af0d0baa95f4520cd05d61031a69a66ec8521bf0/App.js

<FlatList
  horizontal={true}
  data={DATA}
  renderItem={renderFlatList}
  keyExtractor={item => item.toString()}
/>

const Item = ({title}) => (
  <View style={styles.item}>
    <Text style={styles.title}>{title}</Text>
  </View>
);

const renderFlatList = ({item}) => (
  <View>
    <Item title={item.title} />
  </View>
);

data used in flatlist

const DATA = [
  {
    id: '1',
    title: '1 Item',
  },
  {
    id: '2',
    title: '2 Item',
  },
  {
    id: '3',
    title: '3 Item',
  },
  {
    id: '4',
    title: '4 Item',
  },
  {
    id: '5',
    title: 'Fifth Item',
  },
  {
    id: '6',
    title: 'Sixth Item',
  },
  {
    id: '7',
    title: 'Seven Item',
  },
  {
    id: '8',
    title: 'Eight Item',
  },
  {
    id: '9',
    title: 'Nine Item',
  },
  {
    id: '10',
    title: 'Ten Item',
  },
  {
    id: '11',
    title: 'Eleven Item',
  },
  {
    id: '12',
    title: 'Twelve Item',
  },
  {
    id: '13',
    title: 'Thirdteen Item',
  },
  {
    id: '14',
    title: 'Fourteen Item',
  },
  {
    id: '15',
    title: 'Fifthteen Item',
  },
  {
    id: '16',
    title: 'Sixteen Item',
  },
  {
    id: '17',
    title: '17 Item',
  },
  {
    id: '18',
    title: '18 Item',
  },
  {
    id: '19',
    title: '19 Item',
  },
  {
    id: '20',
    title: '20 Item',
  },
  {
    id: '21',
    title: '21 Item',
  },
  {
    id: '22',
    title: '22 Item',
  },
  {
    id: '21',
    title: '23 Item',
  },
  {
    id: '22',
    title: '24 Item',
  },
  {
    id: '22',
    title: '25 Item',
  },
  {
    id: '23',
    title: '26 Item',
  },
  {
    id: '24',
    title: '27 Item',
  },
  {
    id: '25',
    title: '28 Item',
  },
  {
    id: '26',
    title: '29 Item',
  },
];

CLICK TO OPEN VIDEO TESTS

2022-02-25.10-15-30.mp4

[fabOnReact]

TalkBack announces pages and cells with Vertical Flatlist in the Paper Renderer

Expected/Actual Result:
Talkback announces the selected item row and column changes:

1. 2 Item column 2
2. 4 Item row two column 1
3. 11 Item column 2

Expected/Actual Result:
Talkback announces the pages when scrolling down:

Showing Item 10 to 19 of 29

CLICK TO OPEN SOURCECODE

fabOnReact/ReactNativeAwesomeProject@016921e

class App extends React.Component {
  constructor(props) {
    super(props);
  }
  header = () => {
    return (
      <View>
        <Text>POSTS</Text>
      </View>
    );
  };
  render() {
    return (
      <FlatList
        accessible={true}
        accessibilityRole="grid"
        numColumns={3}
        keyExtractor={item => item.id}
        pagingEnabled={false}
        data={DATA}
        renderItem={renderItem}
        keyExtractor={item => item.toString()}
      />
    );
  }
}

const renderItem = ({item, key}) => (
  <View key={key} style={styles.item}>
    <Text key={key}>{item.title}</Text>
  </View>
);

const Item = ({title}) => (
  <View style={styles.item}>
    <Text style={styles.title}>{title}</Text>
  </View>
);

const renderFlatList = ({item}) => (
  <View>
    <Item title={item.title} />
  </View>
);

data used in flatlist

const DATA = [
  {
    id: '1',
    title: '1 Item',
  },
  {
    id: '2',
    title: '2 Item',
  },
  {
    id: '3',
    title: '3 Item',
  },
  {
    id: '4',
    title: '4 Item',
  },
  {
    id: '5',
    title: 'Fifth Item',
  },
  {
    id: '6',
    title: 'Sixth Item',
  },
  {
    id: '7',
    title: 'Seven Item',
  },
  {
    id: '8',
    title: 'Eight Item',
  },
  {
    id: '9',
    title: 'Nine Item',
  },
  {
    id: '10',
    title: 'Ten Item',
  },
  {
    id: '11',
    title: 'Eleven Item',
  },
  {
    id: '12',
    title: 'Twelve Item',
  },
  {
    id: '13',
    title: 'Thirdteen Item',
  },
  {
    id: '14',
    title: 'Fourteen Item',
  },
  {
    id: '15',
    title: 'Fifthteen Item',
  },
  {
    id: '16',
    title: 'Sixteen Item',
  },
  {
    id: '17',
    title: '17 Item',
  },
  {
    id: '18',
    title: '18 Item',
  },
  {
    id: '19',
    title: '19 Item',
  },
  {
    id: '20',
    title: '20 Item',
  },
  {
    id: '21',
    title: '21 Item',
  },
  {
    id: '22',
    title: '22 Item',
  },
  {
    id: '21',
    title: '23 Item',
  },
  {
    id: '22',
    title: '24 Item',
  },
  {
    id: '22',
    title: '25 Item',
  },
  {
    id: '23',
    title: '26 Item',
  },
  {
    id: '24',
    title: '27 Item',
  },
  {
    id: '25',
    title: '28 Item',
  },
  {
    id: '26',
    title: '29 Item',
  },
];

CLICK TO OPEN VIDEO TESTS

2022-02-25.10-35-37.mp4

[fabOnReact]

TalkBack announces pages and cells with Nested Horizontal Flatlist in the rn-tester app

Expected/Actual Result:
Talkback announces the selected item row and column changes:

1. 2 Item column 2
2. 4 Item row two column 1
3. 11 Item column 2

Expected/Actual Result:
Talkback announces the pages when scrolling down:

Showing Item 10 to 19 of 29

CLICK TO OPEN VIDEO TESTS

2022-02-25.15-52-46.mp4

[fabOnReact]

facebook/react-native#33180 (comment)

• adds a View container to each cell or row rendered with the renderItem prop
• {flex: 1} makes the container View take the full width of the parent container
• the container View is accessible and includes the accessibilityCollectionItem information

Screenshots of the two scenarios (with and without {flex: 1})

without {flex: 1}	with {flex: 1}

I moved this logic to VirtualizedList with commits move accessibilityCollectionItem logic to VirtList, Move Logic to virtualized list and removing container View in renderItem callback removing the View container around the cell.

The callback renderItem now has the accessibilityCollectionItem.

Example of the implementation with FlatList

https://github.com/facebook/react-native/blob/c0893d20470e5971eaee5ccc74196dbe8ec84770/packages/rn-tester/js/examples/FlatList/FlatList-multiColumn.js#L143-L156

I added the prop type accessibilityCollectionItem to the View Component, which is used in this case as a wrapper around each FlatList/SectionList cell.

accessibilityCollectionItem in ViewPropTypes

https://github.com/facebook/react-native/blob/c0893d20470e5971eaee5ccc74196dbe8ec84770/Libraries/Components/View/ViewPropTypes.js#L444-L459

The previous solution was adding a wrapper View Component around each cell.
The wrapper in VirtualizedList spans for the entire row (explanation in this post) and does not work with multicolumn flatlist.

I would like to receive your feedback on this solution before investing further time. Is this the solution you had in mind? I can revert this changes or improve it based on your feedback. Thanks

[fabOnReact]

moving _getAccessibilityCollection() to virtualized list

https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/FlatList.js#L685

move to

https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/VirtualizedList.js#L2057-L2079

[fabOnReact]

move accessibilityCollectionItem logic to VirtualizedList

https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/FlatList.js#L615-L640

move to

https://github.com/fabriziobertoglio1987/react-native-notes/blob/ce18d88eb12e50e40b5e6c02bc08da74a5ed5392/Libraries/Lists/VirtualizedList.js#L1983-L2017

The current View container, but represents entire row (2 cells below)

https://github.com/fabriziobertoglio1987/react-native-notes/blob/f49caa490eb2378466bfa3eeb672aa6eab5b65b7/Libraries/Lists/VirtualizedList.js#L2073

2022-03-13.15-40-38.mp4