Repository navigation
Conversation
This change modifies the remaining code that allows for relative RFile and WAL references. The RFiles and WALs that are created today are done with absolute paths, but some of the existing code does not enforce or check that.
| + "Setting this to zero will make bloom filters load in the foreground.", | ||
| "1.3.5"), | ||
| TSERV_MEMDUMP_DIR("tserver.dir.memdump", "/tmp", PropertyType.PATH, """ | ||
| TSERV_MEMDUMP_DIR("tserver.dir.memdump", "file:/tmp", PropertyType.ABSOLUTEPATH, """ |
There was a problem hiding this comment.
Why make this have to be a URI? Its a local file system path and will always be file:/
| * hdfs://foo:6000/accumulo/tables/4/t0/F000.rf -> 4/t0/F000.rf. | ||
| */ | ||
| private String makeRelativeFileReference(String path) { | ||
| Path referencePath = new Path(path); |
There was a problem hiding this comment.
Had a vague memory of performance problems w/ Path and did some digging and found #3509. So it seems calling getParent() on path is slow. Was wondering if the code below could use the path object and call getParent() instead of doing string parsing, but I guess it should not.
| upgradeDataFileCF(key, value, update); | ||
| } else if (family.equals(LogColumnFamily.NAME)) { | ||
| try { | ||
| LogEntry.fromMetaWalEntry(entry); |
There was a problem hiding this comment.
Seems like this is not actually changing anything in the metadata table, its only validating. There are many other columns in the metadata table that we do not validate on upgrade, so this seems like new functionality in upgrade. We could generalize this to upgrade the metadata table and then the upgrade could do a second scan creating tablet metadata object which would do this validation and more. Not something for this PR, but validating the metadata table on upgrade seems like an interesting concept.
| String[] tokens = path.split("/"); | ||
| String[] suffix = new String[4]; | ||
| int suffixLen = 0; | ||
| for (int i = tokens.length - 1; i >= 0 && suffixLen < suffix.length; i--) { |
There was a problem hiding this comment.
This seems much more efficient than the existing makeRelative method which uses a stream. The GC can burn a lot of CPU churning through file refs and in the past CPU has been a bottle neck.
This change modifies the remaining code that allows for relative RFile and WAL references. The RFiles and WALs that are created today are done with absolute paths, but some of the existing code does not enforce or check that.