Is this a XTS bug or a base R bug ? Crashing R

[solvith]

require(xts)
#create a random XTS
XTS<-matrix(rnorm(10000*10),byrow = T,ncol = 10)
XTS<-xts(XTS,order.by = seq(Sys.Date()-7999,Sys.Date(),1))

for (i in 1:10000){
  ifelse(XTS<0,0,1)
}

However , it doesn't crash if i make a smaller XTS:
i.e.,

XTS<-matrix(rnorm(500*10),byrow = T,ncol = 10)

[joshuaulrich]

Please provide sessionInfo() output.

[solvith]

Thanks for the prompt response !
Sure...
basically i just spawn a new R session and run it.. so this is my sessionInfo() before any code is being ran

oops . Here we go
(Edited)

R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] xts_0.9-7 zoo_1.7-13

loaded via a namespace (and not attached):
[1] rsconnect_0.4.3 tools_3.3.1 grid_3.3.1 lattice_0.20-34

[joshuaulrich]

I also need the xts version. Please edit your previous comment with the output from sessionInfo() after calling require(xts).

[joshuaulrich]

I see you're using version 0.9-7, which is not the current development version. In the future, please test on the development version to see if the issue has already been fixed. Also, your example throws an error in the xts() call. Here's an example that runs and does reproduce the issue on version 0.10-0.

require(xts)
nr <- 10000
nc <- 10
set.seed(21)
XTS <- matrix(rnorm(nr*nc), nr, nc)
XTS <- xts(XTS, seq(Sys.Date()-nr+1, Sys.Date(), 1))

for (i in 1:nr) {
  ifelse(XTS<0, 0, 1)
}

 *** caught segfault ***
address 0x49d1000, cause 'memory not mapped'

Traceback:
 1: .Call("_do_subset_xts", x, as.integer(i), as.integer(1:nc), drop, PACKAGE = "xts")
 2: `[.xts`(test, ok)
 3: test[ok]
 4: ifelse(XTS < 0, 0, 1)

The stack trace seems to indicate this is an issue in xts. But if I print the value of i for each iteration of the loop, the crash always happens at iteration 250. I don't see anything obviously odd about XTS[250,], and it doesn't crash if I remove the ifelse call.

I'll have to look at this later, when I can spend more time debugging. Thanks for the report!

[joshuaulrich]

Confirmed this is an xts bug by running the following script under valgrind:

require(xts)
nr <- 10000
nc <- 10
#21 250
#22 10
#23 22
#24 41
set.seed(22)
x <- matrix(rnorm(nr*nc),nr,nc)
x <- xts(x, seq(Sys.Date()-nr+1, Sys.Date(), 1))

for (i in 1:nr) {
  print(i)
  ifelse(x<0,0,1)
}

Here is the output from valgrind:

josh@thinkpad: ~/git/xts
> R -d valgrind -q
==30267== Memcheck, a memory error detector
==30267== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==30267== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==30267== Command: /usr/lib/R/bin/exec/R -q
==30267== 
R> source("xts_ifelse_segfault.R")
Loading required package: xts
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

[1] 1
==30267== Invalid read of size 8
==30267==    at 0xCCCDE68: _do_subset_xts (subset.c:139)
==30267==    by 0x4F0A57F: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F4272E: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44FFD: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==  Address 0x7c9b108 is 0 bytes after a block of size 80,040 alloc'd
==30267==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==30267==    by 0x4F735AC: Rf_allocVector3 (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F12B34: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F12F56: Rf_shallow_duplicate (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4EC5C53: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F3AE2E: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F4215F: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F43DDC: Rf_applyClosure (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F422FC: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42A3A: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F4221B: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F3CDBA: ??? (in /usr/lib/R/lib/libR.so)
==30267== 
==30267== Invalid read of size 4
==30267==    at 0xCCCD454: _do_subset_xts (subset.c:153)
==30267==    by 0x4F0A57F: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F4272E: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44FFD: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==  Address 0xd26a2f8 is 0 bytes after a block of size 400,040 alloc'd
==30267==    at 0x4C2DB8F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==30267==    by 0x4F735AC: Rf_allocVector3 (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F126BC: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F12F56: Rf_shallow_duplicate (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4EC5C53: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F3AE2E: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F4215F: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F43DDC: Rf_applyClosure (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F422FC: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F43DDC: Rf_applyClosure (in /usr/lib/R/lib/libR.so)
==30267== 
==30267== Conditional jump or move depends on uninitialised value(s)
==30267==    at 0x4F664F2: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F76431: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F78108: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F75145: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F35E95: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F4215F: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F43DDC: Rf_applyClosure (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F422FC: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F44E47: ??? (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F42520: Rf_eval (in /usr/lib/R/lib/libR.so)
==30267==    by 0x4F45FB5: ??? (in /usr/lib/R/lib/libR.so)
==30267== 
[1] 2
[1] 3
[1] 4
[1] 5
[1] 6
[1] 7
[1] 8
[1] 9
[1] 10
[1] 11

 *** caught segfault ***
address 0x8fed000, cause 'invalid permissions'

Traceback:
 1: .Call("_do_subset_xts", x, as.integer(i), as.integer(1:nc), drop,     PACKAGE = "xts")
 2: `[.xts`(test, ok)
 3: test[ok]
 4: ifelse(XTS < 0, 0, 1)
 5: eval(expr, envir, enclos)
 6: eval(ei, envir)
 7: withVisible(eval(ei, envir))
 8: source("xts_ifelse_segfault.R")

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace
Selection: 3
==30267== 
==30267== HEAP SUMMARY:
==30267==     in use at exit: 47,621,582 bytes in 13,870 blocks
==30267==   total heap usage: 34,683 allocs, 20,813 frees, 403,247,300 bytes allocated
==30267== 
==30267== LEAK SUMMARY:
==30267==    definitely lost: 0 bytes in 0 blocks
==30267==    indirectly lost: 0 bytes in 0 blocks
==30267==      possibly lost: 0 bytes in 0 blocks
==30267==    still reachable: 47,621,582 bytes in 13,870 blocks
==30267==         suppressed: 0 bytes in 0 blocks
==30267== Rerun with --leak-check=full to see details of leaked memory
==30267== 
==30267== For counts of detected and suppressed errors, rerun with: -v
==30267== Use --track-origins=yes to see where uninitialised values come from
==30267== ERROR SUMMARY: 7129209 errors from 3 contexts (suppressed: 0 from 0)

It appears the problem is that subset.c:139 is not protected by the following block, which precedes the index access in other branches of the if-statement.

if(int_sr[i] == NA_INTEGER)
  error("'i' contains NA");
if(int_sr[i] > nrs || int_sc[j] > ncs)
  error("'i' or 'j' out of range");

The above check would also prevent the invalid read at subset.c:153.