Lazysimpleserde Parameters

您应该尝试在创建表或使用ALTER语句时在表定义级别指定压缩编解码器的用法. CsvStorageHandler parameter is for CSV format. The data is also used outside of Hive. Amazon Athena provides the easiest way to run ad-hoc queries for data in S3 without the need to setup or manage any servers. ) The final DESCRIBE shows the layout of the deeply nested STRUCT type. Subject: last_modified_time and transient_lastDdlTime - what is transient_lastDdlTime for. It can be difficult for you to build an application with access to this tro. recordreader org. In our particular example, partitions are not necessary as the data to be queried is minimal and within the cost parameters. These examples are extracted from open source projects. I have many tables in Hive and suspect size of these tables are causing space issues on HDFS FS. Table partitioning means dividing table data into some parts based on the values of particular columns like date or country, segregate the input records into different files/directories based on date or country. 1 /** 2 * Licensed to the Apache Software Foundation (ASF) under one 3 * or more contributor license agreements. 1推出了Uer Define Function功能,用户可以在Spark SQL 里自定义实际需要的U. In this post, I will. When writing data to a directory, Apache Hive uses LazySimpleSerde for serialization (writing) of results/data. read_csv() sep - Same as pandas. Hive is the datawarehouse layer on top of Hadoop, to provide data summarization, querying and analysis capabilities. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. grants 在hive0. UDF即用户自定函数,注册之后,在sql语句中使用. 0 and later). These examples are extracted from open source projects. In this post I'm going to explain installation of apache hive and some first steps. Yes you will have to put this file in a directory and then create an external table on top of it. LazySimpleSerDe' STORED AS INPUTFORMAT 'com. This is the SerDe for data in CSV, TSV, and custom-delimited formats that Athena uses by default. My View is a simple view which does a select * from the base table. Focus on new technologies and performance tuning. You might need to increase the Map Reduce job time-out parameters as well as a few other parameters in order to schedule a greater number of Mapper processes. If you see something like this: WARN: Missing fields! Expected 6 fields but only got 1. For more information, see skip. skewjoin is that this parameter uses the skew information stored in the metastore to optimize the plan at compile time itself. You can use the DESCRIBE command after the ANALYZE command to display the statistics. There are three ways to describe a table in Hive. read_csv() usecols - Same as pandas. LazySimpleSerDe. 5/spark-warehouse, but actually the warehouse dir is /user/hive/warehouse when. Hive Table for Examples: Lets create below table for demonstrating table related commands. 最佳答案我通过使用describe extended table命令看到例:hive> create table difdelimiter (id int, name string) row format delimited. 1 InputFormat In Hive There are two places where we can spe cify InputFormat i n hive, when creating table and before executing HQL, respectively. (Ashish Thusoo via namit) IMPROVEMENTS (2) HIVE-132. My Question is would the view be Partition aware when a view is queried y the end user or do i need to execute any other commands to be able to use the partitio. If an external table is dropped then only the table metadata is deleted. Ok, that is all Thanks for reading. Our offering of Data Engineering with HADOOP is an intensive amalgamation of all the core components such as MapReduce, HBASE, HDFS, HIVE, SQOOP, PIG, and OOZIE, among others. 1推出了Uer Define Function功能,用户可以在Spark SQL 里自定义实际需要的UDF来处理数据。 因为目前Spark SQL本身支持的函数有限,一些常用的函数都没有,比如len, concatetc 但是使用UDF来自己实现根据业务需要的功能是非常方便的。. SerDeParameters. 格納されているハイブテーブルの区切り記号を知る方法はありますか? Describe extendedを試してみたが無駄です。たくさん検索しましたが、まだ答えは得られていません。. Use this SerDe if your data does not have values enclosed in quotes. In Impala 2. LazySimpleSerDe can be used to read the same data format as MetadataTypedColumnsetSerDe and TCTLSeparatedProtocol. 8、 初始化元数据库 注意:当使用的 hive 是 2. I recently was trying to execute Drill query on Hive table using physical plan for analyzing an issue. We use cookies for various purposes including analytics. You can use the DESCRIBE command after the ANALYZE command to display the statistics. It defines how to read and write data in CSV format. numFiles 3. MFieldSchema" is tagged as >>> "embedded-only" so does not have its own datastore table. Change in Client Connection Timeout The default behavior of client connection timeout changed. Currently, I get an exception: I can show partitons for the table 'a', but can't read or drop it, anybody know why? ps: I am using mysql to store the metadata for hive. As per the documentation : An EXTERNAL TABLE points to any HDFS location for its storage, rather than being stored in a folder specified by the configuration property hive. 以下のddlをクリップボードに貼り付けて、beelineで発行しました。 Points. Table partitioning means dividing table data into some parts based on the values of particular columns like date or country, segregate the input records into different files/directories based on date or country. read_csv() names - Same as pandas. All of those tables are 'internal' tables, ie. For web applications, the command also launches the application in your default browser. The following are top voted examples for showing how to use org. Did you upgrade your Hive installation to Hive 0. getStructFieldRef. 格納されているハイブテーブルの区切り記号を知る方法はありますか? Describe extendedを試してみたが無駄です。たくさん検索しましたが、まだ答えは得られていません。. Specifying this SerDe is optional. I discuss in simple terms how to optimize your AWS Athena configuration for cost effectiveness and performance efficiency, both of which are pillars of the AWS Well Architected Framework. As per the documentation : An EXTERNAL TABLE points to any HDFS location for its storage, rather than being stored in a folder specified by the configuration property hive. In Impala 3. It can be difficult for you to build an application with access to this tro. Problem: Of late, there are many HCC questions on how to create hive table and lineage using REST APIs in Atlas. LazySimpleSerDe: indicates that SerDe(org. In my previous blogpost, I was talking about Schema on Read and Schema on Write advantages and disadvantages. 1 /** 2 * Licensed to the Apache Software Foundation (ASF) under one 3 * or more contributor license agreements. These source code samples are taken from different open source projects. LazySimpleSerDe included by Athena will not support quotes yet. Introduction to AWS Athena. We study what is Skew join in Hive, Parameter of Hive Skew Join, limitations of Skew Join, Skew Join in Hive Examples, Hive Skew Join Working, Skew Join – Use Case, and Tips For Skew Join in Hive. Whilst we could add this query to the Library then copy it where we needed to and make necessary changes in each test, there is an alternative method of reuse we might consider. read_csv() dtype - Same as pandas. 12 per HIVE-4002. OK, I Understand. Saved jobs remember the parameters used to specify a job, so they can be re-executed by invoking the job by its handle. Our offering of Data Engineering with HADOOP is an intensive amalgamation of all the core components such as MapReduce, HBASE, HDFS, HIVE, SQOOP, PIG, and OOZIE, among others. Hive 目前已经成为 Hadoop 大数据离线分析的标准工具,Hive 入门比较容易,但是深入还是有些困难。本次分享目的是带领大家快速入门 Hive,侧. 转载请注明: hive sql分区表 | 学步园 +复制链接. If you are using Aurora 1. set parameter/command ? How to install hive ecosystem under hadoop ? How to use Sqoop Export Statement by using staging table arguments , clear all table arguments and target table in Hadoop ? Export records from Hadoop HDFS to MYSQL table on different scenario using Sqoop Export statement in Hadoop ?. OK, I Understand. I want to know for each of my tables the last time it was modified. 该日志由 serotonin 于6年前发表在综合分类下,最后更新于 2013年09月19日. INSERT OVERWRITE TABLE employees PARTITION (country, state) SELECT , se. A custom SerDe called com. some of my tables don't have last_modified_time in the table parameters but all have transient_lastDdlTime. How to use/configure hive parameter i. Problem: Of late, there are many HCC questions on how to create hive table and lineage using REST APIs in Atlas. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. As follows is an illustration of the problem:. If an external table is dropped then only the table metadata is deleted. LazySimpleSerDe) is inserted, which is used to process text files. ALTER TABLE ADD PARTITION can't handle partition key names that conflict with HQL keywords hive. Ok, that is all Thanks for reading. As usual, the optional IF NOT EXISTS clause will ignore the statement if the table already exists, even when the schema does not match (no warning from Hive). It shows the warehouse dir is file:/root/create/spark/spark-2. SerDeParameters. It can help when the output of the mappers are very small, by reducing the time for shuffle and launching reduce task. Show table and describe results to be read via FetchTask. Enable escaping for the delimiter characters by using the 'ESCAPED BY' clause (such as ESCAPED BY '') Escaping is needed if you want to work with data that can contain these. --connection-name set to the name of the AWS Glue connection you created to point to the Hive metastore. And you can run different types of analytics to better guide decision-making—from dashboards and visualizations to big data processing, real-time. CSE3BDC/CSE5BDC Big Data Management on the Cloud Lecturer: Zhen He Lecture 9 1 Lecture Outline Spark. As follows is an illustration of the problem:. 2 - K value for k-means, in this instance 5. However, LazySimpleSerDe creates Objects in a lazy way, to provide better performance. (For brevity, leave out the ITEM portion of a qualified name when it is not required. xml目前配置如下,3. Donghua's Blog - DBAGlobe Big Data & NoSQL, Information Architecture, Data Management, Governance, etc. LazySimpleSerDe' STORED AS INPUTFORMAT 'com. Using MEP 6. Apache Hive is a data warehouse Infrastructure interface tool to process structured in Hadoop. Question & Answer site for StreamSets big data integration products. For TEXTFILE and the encoding we described in Chapter 3 and repeated in the example above, the SerDe Hive uses is another Java class called org. S3LogDeserializer comes with all EMR AMI's just for parsing these logs. This is the SerDe for data in CSV, TSV, and custom-delimited formats that Athena uses by default. DeprecatedLzoTeUTF-8. How to use/configure hive parameter i. LazySimpleSerDe can treat 'T', 't', 'F', 'f', '1', and '0' as extended, legal boolean literals if the configuration property hive. Use the ANALYZE command to gather statistics and to provide those statistics to the statistics table to optimize the current statement. The default is false, which means only 'TRUE' and 'FALSE' are treated as legal boolean literals. There are hundreds of thousands of partitions and close to 2 million partition parameters-load_catalog_in_background=true in the impala start up options; With so much metadata information, if loading of catalog in the background is set to true, when catalog server starts, it will try to load all the metadata information into memory. I want to know for each of my tables the last time it was modified. Basically, for Serializer/Deserializer, SerDe is an acronym. The API terminology is similar to that used in the web UI: Entity. For TextFile formats, you can specify the SerDe class. where it can infer the partitions to create based on query parameters. 0 we will run 'TRUNCATE TABLE ' as a Hive native command, so it can work. See more of Hive & Pig on Facebook. Support for building tarballs and javadocs (Ashish Thusoo via rmurthy) HIVE-407. SerDeParameters. 转】 Spark SQL UDF使用的更多相关文章. Describes relationship among entities. xml目前配置如下,3. I'm loading a plain text tab delimited file into a Hive (0. 5 / Impala 2. LazySimpleSerDe: indicates that SerDe(org. I recently was trying to execute Drill query on Hive table using physical plan for analyzing an issue. There are much more to learn about, Hive SerDe, let’s begin with its brief Introduction. describe extended employees; name string salary float subordinates array deductions map address struct. LazySimpleSerDe included by Athena will not support quotes yet. Serde parameters: field. O_LINEITEMS and CUSTOMER. objectinspector. >>> 15/02/25 10:31:09 INFO Datastore: The class >>> "org. I want to know for each of my tables the last time it was modified. LazySimpleSerDe, parameters null)], parameters:. * * Also LazySimpleSerDe outputs typed columns instead of treating all columns as * String like MetadataTypedColumnsetSerDe. StructObjectInspector. API Usage Tutorial Cloudera Navigator Concepts. These examples are extracted from open source projects. By voting up you can indicate which examples are most useful and appropriate. To find out more, including how to control cookies, see here. Study Amazon Athena | Creating Tables, Data Formats and Partitions flashcards from Parri Pandian's class online, or in Brainscape's iPhone or Android app. O_LINEITEMS are equivalent. This SerDe is used if you don't specify any SerDe and only specify ROW FORMAT DELIMITED. All big data systems are not made equal, including SQL-on-Hadoop systems like Hive, Polybase and SparkSQL. Thank you Ayub, it's working fine for me, I have one query here, I have one table table5 has columns id, name and age, After inserting the table5 metadata into Atlas I am getting the repeated column I. createtable. Hive is the datawarehouse layer on top of Hadoop, to provide data summarization, querying and analysis capabilities. LazySimpleSerDe included by Athena will not support quotes yet. The sink takes two parameters: the directory and the filename prefix for files written in that directory. LazySimpleSerDe. 5 / Impala 2. The parameters following STORED BY that are corresponding to the built-in storage handlers for csv or tsv files are shown as follows: The com. 13? Unless something in the metadata returned from the Hive 0. Table partitioning means dividing table data into some parts based on the values of particular columns like date or country, segregate the input records into different files/directories based on date or country. numFiles 3. extended_boolean_literal is set to true (Hive 0. All: I apologize in advance if this is common. Both of these argu- ments support the Flume functionality called output bucketing which permits some macro substitutions. For more information, see skip. 基于scala-sdk-2. O_LINEITEMS and CUSTOMER. LazySimpleSerDe. S3://BUCKET_NAME/KEY_NAME) max_result_size - Max number of bytes on each request to S3; header - Same as pandas. ppt from CSE 3BDC at La Trobe University. 在hive创建表的语法中有一个关键字是EXTERNAL,该关键字表示表的类型,在hive中一共有两种类型的表:管理表和托管表(外部表)。. Using the HDFS Connector with Spark Introduction. This is the SerDe for data in CSV, TSV, and custom-delimited formats that Athena uses by default. 解释执行上面语句的数据都放在那些表中(1. Therefore, the DESCRIBE parameters CUSTOMER. The component level setup is optional, but if it is defined, it has higher precedence than the global parameters. identifiers has its default value of “column”. If you see something like this: WARN: Missing fields! Expected 6 fields but only got 1. For instance, in Hive there is a LazyStruct object which is used by the LazySimpleSerDe to represent the deserialized object. I want = to know for each of my tables the last time it was modified. This SerDe is used if you don't specify any SerDe and only specify ROW FORMAT DELIMITED. How to create a table in AWS Athena with background information on HiveQl, as well as multiple methods of creating a table in Athena to suit the end user. Partition Parameters: COLUMN_STATS_ACCURATE true. TextRecordReader The default record reader for reading data from the user scripts. YARN containers running, HIVE store attached (metastore on default port: 9083) I can see my hive…. Spark SQL UDF示例. Here are some examples to show how to pass parameters or user defined variables to hive. Map Join in Hive is also Called Map Side Join in Hive. Thank you Ayub, it's working fine for me, I have one query here, I have one table table5 has columns id, name and age, After inserting the table5 metadata into Atlas I am getting the repeated column I. , those are hive managed tables. Or add following line in the submission script after the command "hive -f hive. (For brevity, leave out the ITEM portion of a qualified name when it is not required. Amazon EMR is flexible - you can run custom applications and code, and define specific compute, memory, storage, and application parameters to optimize your analytic requirements. 0 and later, you can configure SQL standards-based authorization to enable fine grained access control with SQL commands. Although Hive is built to deal with big data, we still cannot ignore the importance of performance. 0-SNAPSHOT-bin-2. read_csv() lineterminator - Same as pandas. recordreader org. 解释执行上面语句的数据都放在那些表中 (1)TBLS(tables)所. HiveではSequenceFileでファイルを読み書きできるが、ファイル内の実データは1行が1つのTextとなる。 Textでなく自分で作ったWritableをデータにしたい場合に、HiveのSerDeを定義する。. What is Hive? What is Metadata? What are the features of Hive? What is the differences Between Hive and HBase? What is Hive Metastore? Wherever (Different Directory) we run hive query, it creates new metastore_db, please explain the reason for it?. st FROM staged_employees se WHERE se. The parameters following STORED BY that are corresponding to the built-in storage handlers for csv or tsv files are shown as follows: The com. >>> 15/02/25 10:31:09 INFO Datastore: The class >>> "org. Currently, I get an exception: I can show partitons for the table 'a', but can't read or drop it, anybody know why? ps: I am using mysql to store the metadata for hive. StructObjectInspector. Or add following line in the submission script after the command "hive -f hive. If you are using Aurora 1. For example, the data files are updated by another process (that does not lock the files. LazySimpleSerDe treats missing fields as NULL. Focus on new technologies and performance tuning. The SQL standards-based authorization mode can be used in conjunction with storage-based authorization on the Metastore server. skewjoin is that this parameter uses the skew information stored in the metastore to optimize the plan at compile time itself. where it can infer the partitions to create based on query parameters. Stay Hungry. Also, if you want to change the column type definition, try “ALTER TABLE REPLACE COLUMNS". According to the document:. (Raghotham Murthy via zshao) HIVE-427. mydb, even when it is not your current working database. S3LogDeserializer comes with all EMR AMI's just for parsing these logs. If there is no skew information in the metadata, this parameter will not have any affect. Using MEP 6. These source code samples are taken from different open source projects. * * Also LazySimpleSerDe outputs typed columns instead of treating all columns as * String like MetadataTypedColumnsetSerDe. 3 and higher, you can specify the name of a complex type column, which takes the form of a dotted path. If this parameter is set to true, Hive delegates the final aggregation stage to a fetch task, possibly decreasing the query time. Still, if any doubt occurs, please share through the comment section. A custom SerDe called com. You can vote up the examples you like and your votes will be used in our system to generate more good examples. When Hive table is created with field delimited as "~" (other than comma) and has the binary column, the data preview on the Hive table fails. This can turn out to be quite a big file so a bit hard to read interactively. 0-SNAPSHOT-bin-2. This SerDe is used if you don't specify any SerDe and only specify ROW FORMAT DELIMITED. 0 and later, you can configure SQL standards-based authorization to enable fine grained access control with SQL commands. Hive syntax errors - NoViableAltException I was trying to load a csv into hive and kept hitting the errors below. Zheng Shao Hi Eric, Most probably there are leading/trailing spaces in the columns that are defined as "int". We can see the Hive tables structures using the Describe commands. Recommend:hadoop - Hive View Partitions ate Field. Using MEP 6. And you can run different types of analytics to better guide decision-making—from dashboards and visualizations to big data processing, real-time. LazySimpleSerDe included by Athena will not support quotes yet. 问题导读1、什么是Uer Define Function功能?2、如何在Spark SQL 里自定义实际需要的UDF来处理数据?3、如何测试concat函数? Spark1. The table is created as shown below. In such a scenario, we can split/partition the data by the month/year or any specified key for faster querying and cost efficiency. As a conclusion, we found that HDFS could be quite suitable for data in the original format. In this post I'm going to explain installation of apache hive and some first steps. Description. This is a known issue and change request (CR) 444305 has been raised for the issue. com Enter your email address: Delivered by FeedBurner. Below parameter needs to be set to enable skew join. Follow these steps to find out if your tables don't have the TableType attribute that Amazon QuickSight needs for the Athena connector to work. --connection-name set to the name of the AWS Glue connection you created to point to the Hive metastore. 文章转载自:量子恒道官方博客HIVE的CTAS用法探究Postedbylengchuanon2011年4月29日LeaveacommentGotocomments最近在使用ADM系统的时候遇到一个问题,ADM在自动将HIVEQL. j'ai beaucoup cherché, pas encore. Execution and testing process: 1. My View is a simple view which does a select * from the base table. Refer to Hive SerDe for more information. Hi again! Here is another issue I don't understand why happens: The size of the table doubles if I load the data with INSERT OVERWRITE vs LOAD. But ORC file format has some problems when working with Cloudera - Oracle distro. The DESCRIBE statement displays metadata about a table, such as the column names and their data types. If this is a HQE type of access a query is issued to Hive that will build a TEXT formatted result table in HDFS. 2 and lower, client waited indefinitely to open the new session if the maximum number of threads specified by --fe_service_threads has been allocated. Create lzo related tables: (validation process, negligible) create external table test_lzo( id int )partitioned by(`date_par` string) ROW FORMAT SERDE 'org. I have searched a lot, not yet getting the answer. i have installed spark on ubuntu without HDFS. Download Of course first we have to download everything: hive and hadoop (hive will not work without it). Basically, that feature is what we call Map join in Hive. There are hundreds of thousands of partitions and close to 2 million partition parameters-load_catalog_in_background=true in the impala start up options; With so much metadata information, if loading of catalog in the background is set to true, when catalog server starts, it will try to load all the metadata information into memory. Tips and tricks to manage your Hadoop cluster in Windows Azure. j'ai beaucoup cherché, pas encore. 从mysql转数据到hive中,本想用parquet格式,但是一直都没有成功,提示. Clone via HTTPS Clone with Git or checkout with SVN using the repository's web address. x 之前的版本,不做初始化也是 OK 的,当 hive 第一次启动的 时候会自动进行初始化,只不过会不会生成足够多的元数据库中的表。. This introduction to AWS Athena gives a brief overview of what what AWS Athena is and some potential use cases. I want to know for each of my tables the last time it was modified. 13? Unless something in the metadata returned from the Hive 0. Hive Table Related Commands. When creating the new table, the location parameter can be specified. We use cookies for various purposes including analytics. Question & Answer site for StreamSets big data integration products. read_csv() dtype - Same as pandas. 创建数据库名为hive 2. You can use any of the following options based on your environment and needs: Clean up the input data (strings) before import. Although Hive is built to deal with big data, we still cannot ignore the importance of performance. O_LINEITEMS and CUSTOMER. LazySimpleSerDe can treat 'T', 't', 'F', 'f', '1', and '0' as extended, legal boolean literals if the configuration property hive. LanguageManual DDL - Apache Hive - Apache Software Foundation テーブル定義の確認方法 HIVEで作成したテーブルのファイルフォーマットや構成を確認したい場合 hive > desc [ extended / formatted ] テーブル名 extende…. LazySimpleSerDe for CSV, TSV, and Custom-Delimited Files. There are hundreds of thousands of partitions and close to 2 million partition parameters-load_catalog_in_background=true in the impala start up options; With so much metadata information, if loading of catalog in the background is set to true, when catalog server starts, it will try to load all the metadata information into memory. LazySimpleSerDe included by Athena will not support quotes yet. As follows is an illustration of the problem:. 13 metastore is different, the Hive 0. But when this is true, it uses a fetch task for the aggregation and no reducer is ran. Still, if any doubt occurs, please share through the comment section. The execution was getting failed with below error. Show table and describe results to be read via FetchTask. In this article we learned. read_csv() sep - Same as pandas. Subject: last_modified_time and transient_lastDdlTime - what is transient_lastDdlTime for. deserialize() to perform deserialization of the record. Here are the examples of the python api pandas. If the limit n is greater than zero then the pattern will be applied at most n - 1 times, the array's length will be no greater than n , and the array's last entry will contain all input beyond the last matched delimiter. By continuing to use Pastebin, you agree to our use of cookies as described in the Cookies Policy. By continuing to use this website, you agree to their use. Hive non creare le partizioni per voi in questo modo. some of my tables don't have last_modified_time in the table parameters but all have transient_lastDdlTime. An entity can be uniquely identified by its identity. * * Also LazySimpleSerDe outputs typed columns instead of treating all columns as * String like MetadataTypedColumnsetSerDe. Amazon EMR is flexible - you can run custom applications and code, and define specific compute, memory, storage, and application parameters to optimize your analytic requirements. We change the value from 3 second to 24 second in our case, because this value should not be too large of the timeout retransmission. INSERT OVERWRITE TABLE employees PARTITION (country, state) SELECT , se. Abstract data structure that describes structural features of any entity. 2 and lower, client waited indefinitely to open the new session if the maximum number of threads specified by --fe_service_threads has been allocated. What will you learn from this hive tutorial? This hadoop hive tutorial shows how to use various Hive commands in HQL to perform various operations like creating a table in hive, deleting a table in hive, altering a table in hive, etc. 0 and later). We use cookies for various purposes including analytics. There are three ways to describe a table in Hive. It was added in Hive 0. Problem: Of late, there are many HCC questions on how to create hive table and lineage using REST APIs in Atlas. 文件文件(TEXTFILE)格式是Hive中默认的存储格式,在创建Hive 表时如果不指定存储格式即为文本文件。. Each key and its corresponding value in the property list is a string. 행동의 씨앗을 뿌리면 습관의 열매가 열리고, 습관의 씨앗을 뿌리면 성격의 열매가 열리고, 성격의 씨앗을 뿌리면 운명의 열매가 열린다. LazySimpleSerDe. some of my = tables don't have last_modified_time in the table parameters but = all have transient_lastDdlTime. The engine then invokes Serde. There are hundreds of thousands of partitions and close to 2 million partition parameters-load_catalog_in_background=true in the impala start up options; With so much metadata information, if loading of catalog in the background is set to true, when catalog server starts, it will try to load all the metadata information into memory. getStructFieldRef.