首页 / 浏览问题 / 组件GIS / 问题详情
spark模块数据存入Hbase数据源报错
6EXP 2020年10月28日

执行内容:

 FeatureRDDProviderFactory(getHBaseProviderParam()).save(result, getHBaseProviderParam(), phonegjLineName)

报错内容如下:

Exception in thread "main" java.lang.IllegalArgumentException: The simple feature type contains attribute name(s) that are reserved words: ID. You may override this check by putting Boolean.TRUE into the SimpleFeatureType user data under the key 'override.reserved.words' before calling createSchema, or by setting the system property 'override.reserved.words' to 'true', however it may cause errors with some functionality.
    at org.locationtech.geomesa.utils.index.ReservedWordCheck$.validateAttributeNames(GeoMesaSchemaValidator.scala:56)
    at org.locationtech.geomesa.utils.index.GeoMesaSchemaValidator$.validate(GeoMesaSchemaValidator.scala:30)
    at org.locationtech.geomesa.index.geotools.MetadataBackedDataStore.validateNewSchema(MetadataBackedDataStore.scala:57)
    at org.locationtech.geomesa.index.geotools.GeoMesaDataStore.validateNewSchema(GeoMesaDataStore.scala:112)
    at org.locationtech.geomesa.hbase.data.HBaseDataStore.validateNewSchema(HBaseDataStore.scala:52)
    at org.locationtech.geomesa.index.geotools.MetadataBackedDataStore.createSchema(MetadataBackedDataStore.scala:138)
    at org.locationtech.geomesa.hbase.data.HBaseDataStore.createSchema(HBaseDataStore.scala:61)
    at com.supermap.bdt.rddprovider.hbase.GeoMesaHBaseFeatureRDDProvider.createSchema(GeoMesaHBaseFeatureRDDProvider.scala:294)
    at com.supermap.bdt.FeatureRDDProvider$class.save(FeatureRDDProvider.scala:72)
    at com.supermap.bdt.rddprovider.hbase.GeoMesaHBaseFeatureRDDProvider.save(GeoMesaHBaseFeatureRDDProvider.scala:35)
    at com.cloudoforce.spark.app.analysis.FollowAnalysisApp$.pointToLineGj1(FollowAnalysisApp.scala:760)
    at com.cloudoforce.spark.app.analysis.FollowAnalysisApp$.main(FollowAnalysisApp.scala:147)
    at com.cloudoforce.spark.app.analysis.FollowAnalysisApp.main(FollowAnalysisApp.scala)
2020-10-28 18:21:56 INFO  SparkContext:54 - Invoking stop() from shutdown hook

其实我在创建SimpleFeatureType已经加入了userdata的设置

 public static SimpleFeatureType getFeatureType(String typeName, String geoType) {

        SimpleFeatureTypeBuilder builder = new SimpleFeatureTypeBuilder();
        builder.setName(typeName);
        builder.setCRS(DefaultGeographicCRS.WGS84);// <- Coordinate reference system
        JSONObject ajson = DatasetSetting.getDvSetting(geoType);

        String geo = ajson.getString("geo");
        String colName = ajson.getString("colName");
        String colType = ajson.getString("colType");
        String colLength = ajson.getString("colLength");

        //判断空间类型
        if(!"table".equals(geo)){
            if ("point".equals(geo)) builder.userData("override.reserved.words","true").add("smgeometry", Point.class);
            else if ("polyline".equals(geo)) builder.userData("override.reserved.words","true").add("smgeometry", LineString.class);
            else if ("polygon".equals(geo)) builder.userData("override.reserved.words","true").add("smgeometry", Polygon.class);
        }

        String[] colNames = colName.split(",");
        String[] colTypes = colType.split(",");
        String[] colLengths = colLength.split(",");

        for (int i = 0; i < colNames.length; i++) {
            String aColName = colNames[i];
            String aColType = colTypes[i];
            int aColLength = Integer.parseInt(colLengths[i]);
            if ("text".equals(aColType)) {
                builder.userData("override.reserved.words","true").length(aColLength).add(aColName, String.class);
            } else if ("int16".equals(aColType)) {
                builder.userData("override.reserved.words","true").length(aColLength).add(aColName, Short.class);
            } else if ("int32".equals(aColType)) {
                builder.userData("override.reserved.words","true").length(aColLength).add(aColName, Integer.class);
            } else if ("int64".equals(aColType)) {
                builder.userData("override.reserved.words","true").length(aColLength).add(aColName, Long.class);
            } else if ("double".equals(aColType)) {
                builder.userData("override.reserved.words","true").minOccurs(0).maxOccurs(2).nillable(false).add(aColName,Double.class);
            } else if ("date".equals(aColType)) {
                builder.userData("override.reserved.words","true").add(aColName, Timestamp.class);
            }
        }

        // build the type
        return builder.buildFeatureType();
    }

1个回答

数据里存在系统保留关键词,需要在hbase连接参数设置下,如下参数

params.put(HBaseFeatureRDDProviderParams.RESERVED_WORDS, true)
1,430EXP 2020年10月29日
...