2022 Current Associate-Developer-Apache-Spark dumps Preparation through Our Practice Test [Q52-Q77]

2022 Current Associate-Developer-Apache-Spark dumps Preparation through Our Practice Test

100% Reliable Microsoft Associate-Developer-Apache-Spark Exam Dumps Test Pdf Exam Material

The best way to study for a Databricks Associate Developer Apache Spark Exam is by getting as many

Many of the questions you will face when taking the Databricks Associate Developer exam are based on real-world scenarios that can only be simulated in the Databricks environment. Our team of subject matter experts have designed a series of practice exams that will help you prepare for this exam. With our online practice exams, you can simulate the actual Databricks environment and learn from your mistakes while working your way through the questions. Databricks Associate Developer Apache Spark exam dumps will save your time and money.

We developed the online test platform because we wanted to make sure that you could practice on your own schedule. You can take the test anytime, and you can retake it as many times as you like.

In conclusion, the best way to learn something is to practice it. If you're a beginner, it's recommended that you start with the free practice exams available on our website. Once you've mastered the fundamentals, you can move on to the official Databricks Associate Developer Apache Spark exam prep materials. They come with an accompanying practice test. You'll get the chance to test your knowledge before the actual exam. This will help you know if you have what it takes to pass the real exam. If you do, you can skip the official exam prep materials and focus on learning the concepts covered in the practice test.

Understand why Databricks experts work only for top consulting companies

Many business owners are excited about the idea of using Databricks for their business. However, the fact is that Databricks experts can only be found at the top consulting companies. But why? What makes these companies so special? And what can you do to get the best Databricks experts for your business?

These consulting companies are the best because they have the most experienced and skilled consultants. They know how to use Databricks to help their clients. They also know the best practices for using Databricks. They know how to make the most out of Databricks. Databricks Associate Developer Apache Spark exam dumps are the best way to pass this exam.

Databricks is an amazing tool for data scientists. It is a great tool for data engineers as well. They are using it to build powerful applications for different industries. They are also using it to solve real-world problems. This is a great way to solve real-world problems. Data scientists are the most important people in any company. They are the ones who can bring the most value to any organization.

For more info read the reference:

Databricks Associate Developer Apache Spark Exam

NEW QUESTION 52
Which of the following code blocks shuffles DataFrame transactionsDf, which has 8 partitions, so that it has
10 partitions?

A. transactionsDf.repartition(transactionsDf.rdd.getNumPartitions()+2)
B. transactionsDf.coalesce(transactionsDf.getNumPartitions()+2)
C. transactionsDf.repartition(transactionsDf.getNumPartitions()+2)
D. transactionsDf.coalesce(10)
E. transactionsDf.repartition(transactionsDf._partitions+2)

Answer: A

Explanation:
Explanation
transactionsDf.repartition(transactionsDf.rdd.getNumPartitions()+2)
Correct. The repartition operator is the correct one for increasing the number of partitions. calling getNumPartitions() on DataFrame.rdd returns the current number of partitions.
transactionsDf.coalesce(10)
No, after this command transactionsDf will continue to only have 8 partitions. This is because coalesce() can only decreast the amount of partitions, but not increase it.
transactionsDf.repartition(transactionsDf.getNumPartitions()+2)
Incorrect, there is no getNumPartitions() method for the DataFrame class.
transactionsDf.coalesce(transactionsDf.getNumPartitions()+2)
Wrong, coalesce() can only be used for reducing the number of partitions and there is no getNumPartitions() method for the DataFrame class.
transactionsDf.repartition(transactionsDf._partitions+2)
No, DataFrame has no _partitions attribute. You can find out the current number of partitions of a DataFrame with the DataFrame.rdd.getNumPartitions() method.
More info: pyspark.sql.DataFrame.repartition - PySpark 3.1.2 documentation, pyspark.RDD.getNumPartitions - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 3

NEW QUESTION 53
Which of the following describes tasks?

A. Tasks transform jobs into DAGs.
B. A task is a collection of rows.
C. A task is a command sent from the driver to the executors in response to a transformation.
D. A task is a collection of slots.
E. Tasks get assigned to the executors by the driver.

Answer: E

Explanation:
Explanation
Tasks get assigned to the executors by the driver.
Correct! Or, in other words: Executors take the tasks that they were assigned to by the driver, run them over partitions, and report the their outcomes back to the driver.
Tasks transform jobs into DAGs.
No, this statement disrespects the order of elements in the Spark hierarchy. The Spark driver transforms jobs into DAGs. Each job consists of one or more stages. Each stage contains one or more tasks.
A task is a collection of rows.
Wrong. A partition is a collection of rows. Tasks have little to do with a collection of rows. If anything, a task processes a specific partition.
A task is a command sent from the driver to the executors in response to a transformation.
Incorrect. The Spark driver does not send anything to the executors in response to a transformation, since transformations are evaluated lazily. So, the Spark driver would send tasks to executors only in response to actions.
A task is a collection of slots.
No. Executors have one or more slots to process tasks and each slot can be assigned a task.

NEW QUESTION 54
Which of the following statements about Spark's DataFrames is incorrect?

A. Spark's DataFrames are equal to Python's DataFrames.
B. Data in DataFrames is organized into named columns.
C. The data in DataFrames may be split into multiple chunks.
D. RDDs are at the core of DataFrames.
E. Spark's DataFrames are immutable.

Answer: A

Explanation:
Explanation
Spark's DataFrames are equal to Python's or R's DataFrames.
No, they are not equal. They are only similar. A major difference between Spark and Python is that Spark's DataFrames are distributed, whereby Python's are not.

NEW QUESTION 55
Which of the following statements about Spark's execution hierarchy is correct?

A. In Spark's execution hierarchy, tasks are one layer above slots.
B. In Spark's execution hierarchy, manifests are one layer above jobs.
C. In Spark's execution hierarchy, executors are the smallest unit.
D. In Spark's execution hierarchy, a stage comprises multiple jobs.
E. In Spark's execution hierarchy, a job may reach over multiple stage boundaries.

Answer: E

Explanation:
Explanation
In Spark's execution hierarchy, a job may reach over multiple stage boundaries.
Correct. A job is a sequence of stages, and thus may reach over multiple stage boundaries.
In Spark's execution hierarchy, tasks are one layer above slots.
Incorrect. Slots are not a part of the execution hierarchy. Tasks are the lowest layer.
In Spark's execution hierarchy, a stage comprises multiple jobs.
No. It is the other way around - a job consists of one or multiple stages.
In Spark's execution hierarchy, executors are the smallest unit.
False. Executors are not a part of the execution hierarchy. Tasks are the smallest unit!
In Spark's execution hierarchy, manifests are one layer above jobs.
Wrong. Manifests are not a part of the Spark ecosystem.

NEW QUESTION 56
Which of the following describes how Spark achieves fault tolerance?

A. Spark is only fault-tolerant if this feature is specifically enabled via the spark.fault_recovery.enabled property.
B. Spark builds a fault-tolerant layer on top of the legacy RDD data system, which by itself is not fault tolerant.
C. If an executor on a worker node fails while calculating an RDD, that RDD can be recomputed by another executor using the lineage.
D. Due to the mutability of DataFrames after transformations, Spark reproduces them using observed lineage in case of worker node failure.
E. Spark helps fast recovery of data in case of a worker fault by providing the MEMORY_AND_DISK storage level option.

Answer: C

Explanation:
Explanation
Due to the mutability of DataFrames after transformations, Spark reproduces them using observed lineage in case of worker node failure.
Wrong - Between transformations, DataFrames are immutable. Given that Spark also records the lineage, Spark can reproduce any DataFrame in case of failure. These two aspects are the key to understanding fault tolerance in Spark.
Spark builds a fault-tolerant layer on top of the legacy RDD data system, which by itself is not fault tolerant.
Wrong. RDD stands for Resilient Distributed Dataset and it is at the core of Spark and not a "legacy system".
It is fault-tolerant by design.
Spark helps fast recovery of data in case of a worker fault by providing the MEMORY_AND_DISK storage level option.
This is not true. For supporting recovery in case of worker failures, Spark provides "_2", "_3", and so on, storage level options, for example MEMORY_AND_DISK_2. These storage levels are specifically designed to keep duplicates of the data on multiple nodes. This saves time in case of a worker fault, since a copy of the data can be used immediately, vs. having to recompute it first.
Spark is only fault-tolerant if this feature is specifically enabled via the spark.fault_recovery.enabled property.
No, Spark is fault-tolerant by design.

NEW QUESTION 57
The code block displayed below contains an error. The code block should return a DataFrame in which column predErrorAdded contains the results of Python function add_2_if_geq_3 as applied to numeric and nullable column predError in DataFrame transactionsDf. Find the error.
Code block:
1.def add_2_if_geq_3(x):
2. if x is None:
3. return x
4. elif x >= 3:
5. return x+2
6. return x
7.
8.add_2_if_geq_3_udf = udf(add_2_if_geq_3)
9.
10.transactionsDf.withColumnRenamed("predErrorAdded", add_2_if_geq_3_udf(col("predError")))

A. UDFs are only available through the SQL API, but not in the Python API as shown in the code block.
B. The operator used to adding the column does not add column predErrorAdded to the DataFrame.
C. The Python function is unable to handle null values, resulting in the code block crashing on execution.
D. Instead of col("predError"), the actual DataFrame with the column needs to be passed, like so transactionsDf.predError.
E. The udf() method does not declare a return type.

Answer: B

Explanation:
Explanation
Correct code block:
def add_2_if_geq_3(x):
if x is None:
return x
elif x >= 3:
return x+2
return x
add_2_if_geq_3_udf = udf(add_2_if_geq_3)
transactionsDf.withColumn("predErrorAdded", add_2_if_geq_3_udf(col("predError"))).show() Instead of withColumnRenamed, you should use the withColumn operator.
The udf() method does not declare a return type.
It is fine that the udf() method does not declare a return type, this is not a required argument. However, the default return type is StringType. This may not be the ideal return type for numeric, nullable data - but the code will run without specified return type nevertheless.
The Python function is unable to handle null values, resulting in the code block crashing on execution.
The Python function is able to handle null values, this is what the statement if x is None does.
UDFs are only available through the SQL API, but not in the Python API as shown in the code block.
No, they are available through the Python API. The code in the code block that concerns UDFs is correct.
Instead of col("predError"), the actual DataFrame with the column needs to be passed, like so transactionsDf.predError.
You may choose to use the transactionsDf.predError syntax, but the col("predError") syntax is fine.

NEW QUESTION 58
Which of the following code blocks saves DataFrame transactionsDf in location /FileStore/transactions.csv as a CSV file and throws an error if a file already exists in the location?

A. transactionsDf.write.format("csv").mode("error").path("/FileStore/transactions.csv")
B. transactionsDf.write("csv").mode("error").save("/FileStore/transactions.csv")
C. transactionsDf.write.format("csv").mode("error").save("/FileStore/transactions.csv")
D. transactionsDf.write.save("/FileStore/transactions.csv")
E. transactionsDf.write.format("csv").mode("ignore").path("/FileStore/transactions.csv")

Answer: C

Explanation:
Explanation
Static notebook | Dynamic notebook: See test 1
(https://flrs.github.io/spark_practice_tests_code/#1/28.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

NEW QUESTION 59
The code block shown below should return all rows of DataFrame itemsDf that have at least 3 items in column itemNameElements. Choose the answer that correctly fills the blanks in the code block to accomplish this.
Example of DataFrame itemsDf:
1.+------+----------------------------------+-------------------+------------------------------------------+
2.|itemId|itemName |supplier |itemNameElements |
3.+------+----------------------------------+-------------------+------------------------------------------+
4.|1 |Thick Coat for Walking in the Snow|Sports Company Inc.|[Thick, Coat, for, Walking, in, the, Snow]|
5.|2 |Elegant Outdoors Summer Dress |YetiX |[Elegant, Outdoors, Summer, Dress] |
6.|3 |Outdoors Backpack |Sports Company Inc.|[Outdoors, Backpack] |
7.+------+----------------------------------+-------------------+------------------------------------------+ Code block:
itemsDf.__1__(__2__(__3__)__4__)

A. 1. filter
2. size
3. "itemNameElements"
4. >=3
(Correct)
B. 1. select
2. size
3. "itemNameElements"
4. >3
C. 1. select
2. count
3. col("itemNameElements")
4. >3
D. 1. select
2. count
3. "itemNameElements"
4. >3
E. 1. filter
2. count
3. itemNameElements
4. >=3

Answer: A

Explanation:
Explanation
Correct code block:
itemsDf.filter(size("itemNameElements")>3)
Output of code block:
+------+----------------------------------+-------------------+------------------------------------------+
|itemId|itemName |supplier |itemNameElements |
+------+----------------------------------+-------------------+------------------------------------------+
|1 |Thick Coat for Walking in the Snow|Sports Company Inc.|[Thick, Coat, for, Walking, in, the, Snow]|
|2 |Elegant Outdoors Summer Dress |YetiX |[Elegant, Outdoors, Summer, Dress] |
+------+----------------------------------+-------------------+------------------------------------------+ The big difficulty with this question is in knowing the difference between count and size (refer to documentation below). size is the correct function to choose here since it returns the number of elements in an array on a per-row basis.
The other consideration for solving this question is the difference between select and filter. Since we want to return the rows in the original DataFrame, filter is the right choice. If we would use select, we would simply get a single-column DataFrame showing which rows match the criteria, like so:
+----------------------------+
|(size(itemNameElements) > 3)|
+----------------------------+
|true |
|true |
|false |
+----------------------------+
More info:
Count documentation: pyspark.sql.functions.count - PySpark 3.1.1 documentation Size documentation: pyspark.sql.functions.size - PySpark 3.1.1 documentation Static notebook | Dynamic notebook: See test 1

NEW QUESTION 60
Which of the following code blocks creates a new one-column, two-row DataFrame dfDates with column date of type timestamp?

A. 1.dfDates = spark.createDataFrame([("23/01/2022 11:28:12",),("24/01/2022 10:58:34",)], ["date"])
B. 1.dfDates = spark.createDataFrame(["23/01/2022 11:28:12","24/01/2022 10:58:34"], ["date"])
2.dfDates = dfDates.withColumn("date", to_timestamp("dd/MM/yyyy HH:mm:ss", "date"))
C. 1.dfDates = spark.createDataFrame(["23/01/2022 11:28:12","24/01/2022 10:58:34"], ["date"])
2.dfDates = dfDates.withColumnRenamed("date", to_datetime("date", "yyyy-MM-dd HH:mm:ss"))
D. 1.dfDates = spark.createDataFrame([("23/01/2022 11:28:12",),("24/01/2022 10:58:34",)], ["date"])
2.dfDates = dfDates.withColumn("date", to_timestamp("date", "dd/MM/yyyy HH:mm:ss"))
E. 1.dfDates = spark.createDataFrame([("23/01/2022 11:28:12",),("24/01/2022 10:58:34",)], ["date"])
2.dfDates = dfDates.withColumnRenamed("date", to_timestamp("date", "yyyy-MM-dd HH:mm:ss"))

Answer: D

Explanation:
Explanation
This question is tricky. Two things are important to know here:
First, the syntax for createDataFrame: Here you need a list of tuples, like so: [(1,), (2,)]. To define a tuple in Python, if you just have a single item in it, it is important to put a comma after the item so that Python interprets it as a tuple and not just a normal parenthesis.
Second, you should understand the to_timestamp syntax. You can find out more about it in the documentation linked below.
For good measure, let's examine in detail why the incorrect options are wrong:
dfDates = spark.createDataFrame([("23/01/2022 11:28:12",),("24/01/2022 10:58:34",)], ["date"]) This code snippet does everything the question asks for - except that the data type of the date column is a string and not a timestamp. When no schema is specified, Spark sets the string data type as default.
dfDates = spark.createDataFrame(["23/01/2022 11:28:12","24/01/2022 10:58:34"], ["date"]) dfDates = dfDates.withColumn("date", to_timestamp("dd/MM/yyyy HH:mm:ss", "date")) In the first row of this command, Spark throws the following error: TypeError: Can not infer schema for type:
<class 'str'>. This is because Spark expects to find row information, but instead finds strings. This is why you need to specify the data as tuples. Fortunately, the Spark documentation (linked below) shows a number of examples for creating DataFrames that should help you get on the right track here.
dfDates = spark.createDataFrame([("23/01/2022 11:28:12",),("24/01/2022 10:58:34",)], ["date"]) dfDates = dfDates.withColumnRenamed("date", to_timestamp("date", "yyyy-MM-dd HH:mm:ss")) The issue with this answer is that the operator withColumnRenamed is used. This operator simply renames a column, but it has no power to modify its actual content. This is why withColumn should be used instead. In addition, the date format yyyy-MM-dd HH:mm:ss does not reflect the format of the actual timestamp: "23/01/2022 11:28:12".
dfDates = spark.createDataFrame(["23/01/2022 11:28:12","24/01/2022 10:58:34"], ["date"]) dfDates = dfDates.withColumnRenamed("date", to_datetime("date", "yyyy-MM-dd HH:mm:ss")) Here, withColumnRenamed is used instead of withColumn (see above). In addition, the rows are not expressed correctly - they should be written as tuples, using parentheses. Finally, even the date format is off here (see above).
More info: pyspark.sql.functions.to_timestamp - PySpark 3.1.2 documentation and pyspark.sql.SparkSession.createDataFrame - PySpark 3.1.1 documentation Static notebook | Dynamic notebook: See test 2

NEW QUESTION 61
Which of the following describes the conversion of a computational query into an execution plan in Spark?

A. The executed physical plan depends on a cost optimization from a previous stage.
B. The catalog assigns specific resources to the optimized memory plan.
C. Depending on whether DataFrame API or SQL API are used, the physical plan may differ.
D. Spark uses the catalog to resolve the optimized logical plan.
E. The catalog assigns specific resources to the physical plan.

Answer: A

Explanation:
Explanation
The executed physical plan depends on a cost optimization from a previous stage.
Correct! Spark considers multiple physical plans on which it performs a cost analysis and selects the final physical plan in accordance with the lowest-cost outcome of that analysis. That final physical plan is then executed by Spark.
Spark uses the catalog to resolve the optimized logical plan.
No. Spark uses the catalog to resolve the unresolved logical plan, but not the optimized logical plan. Once the unresolved logical plan is resolved, it is then optimized using the Catalyst Optimizer.
The optimized logical plan is the input for physical planning.
The catalog assigns specific resources to the physical plan.
No. The catalog stores metadata, such as a list of names of columns, data types, functions, and databases.
Spark consults the catalog for resolving the references in a logical plan at the beginning of the conversion of the query into an execution plan. The result is then an optimized logical plan.
Depending on whether DataFrame API or SQL API are used, the physical plan may differ.
Wrong - the physical plan is independent of which API was used. And this is one of the great strengths of Spark!
The catalog assigns specific resources to the optimized memory plan.
There is no specific "memory plan" on the journey of a Spark computation.
More info: Spark's Logical and Physical plans ... When, Why, How and Beyond. | by Laurent Leturgez | datalex | Medium

NEW QUESTION 62
Which of the following describes Spark's way of managing memory?

A. As a general rule for garbage collection, Spark performs better on many small objects than few big objects.
B. Storage memory is used for caching partitions derived from DataFrames.
C. Spark uses a subset of the reserved system memory.
D. Disabling serialization potentially greatly reduces the memory footprint of a Spark application.
E. Spark's memory usage can be divided into three categories: Execution, transaction, and storage.

Answer: B

Explanation:
Explanation
Spark's memory usage can be divided into three categories: Execution, transaction, and storage.
No, it is either execution or storage.
As a general rule for garbage collection, Spark performs better on many small objects than few big objects.
No, Spark's garbage collection runs faster on fewer big objects than many small objects.
Disabling serialization potentially greatly reduces the memory footprint of a Spark application.
The opposite is true - serialization reduces the memory footprint, but may impact performance in a negative way.
Spark uses a subset of the reserved system memory.
No, the reserved system memory is separate from Spark memory. Reserved memory stores Spark's internal objects.
More info: Tuning - Spark 3.1.2 Documentation, Spark Memory Management | Distributed Systems Architecture, Learning Spark, 2nd Edition, Chapter 7

NEW QUESTION 63
The code block displayed below contains an error. The code block is intended to join DataFrame itemsDf with the larger DataFrame transactionsDf on column itemId. Find the error.
Code block:
transactionsDf.join(itemsDf, "itemId", how="broadcast")

A. The syntax is wrong, how= should be removed from the code block.
B. Spark will only perform the broadcast operation if this behavior has been enabled on the Spark cluster.
C. The join method should be replaced by the broadcast method.
D. The larger DataFrame transactionsDf is being broadcasted, rather than the smaller DataFrame itemsDf.
E. broadcast is not a valid join type.

Answer: E

Explanation:
Explanation
broadcast is not a valid join type.
Correct! The code block should read transactionsDf.join(broadcast(itemsDf), "itemId"). This would imply an inner join (this is the default in DataFrame.join()), but since the join type is not given in the question, this would be a valid choice.
The larger DataFrame transactionsDf is being broadcasted, rather than the smaller DataFrame itemsDf.
This option does not apply here, since the syntax around broadcasting is incorrect.
Spark will only perform the broadcast operation if this behavior has been enabled on the Spark cluster.
No, it is enabled by default, since the spark.sql.autoBroadcastJoinThreshold property is set to 10 MB by default. If that property would be set to -1, then broadcast joining would be disabled.
More info: Performance Tuning - Spark 3.1.1 Documentation (https://bit.ly/3gCz34r) The join method should be replaced by the broadcast method.
No, DataFrame has no broadcast() method.
The syntax is wrong, how= should be removed from the code block.
No, having the keyword argument how= is totally acceptable.

NEW QUESTION 64
The code block displayed below contains an error. The code block is intended to perform an outer join of DataFrames transactionsDf and itemsDf on columns productId and itemId, respectively.
Find the error.
Code block:
transactionsDf.join(itemsDf, [itemsDf.itemId, transactionsDf.productId], "outer")

A. The term [itemsDf.itemId, transactionsDf.productId] should be replaced by itemsDf.itemId == transactionsDf.productId.
B. The join type needs to be appended to the join() operator, like join().outer() instead of listing it as the last argument inside the join() call.
C. The "outer" argument should be eliminated from the call and join should be replaced by joinOuter.
D. The "outer" argument should be eliminated, since "outer" is the default join type.
E. The term [itemsDf.itemId, transactionsDf.productId] should be replaced by itemsDf.col("itemId") == transactionsDf.col("productId").

Answer: A

Explanation:
Explanation
Correct code block:
transactionsDf.join(itemsDf, itemsDf.itemId == transactionsDf.productId, "outer") Static notebook | Dynamic notebook: See test 1 (https://flrs.github.io/spark_practice_tests_code/#1/33.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

A. 1. size
2. regexp_replace
3. lower
4. "itemName"
5. alias
B. 1. length
2. regexp_replace
3. lower
4. col("itemName")
5. alias
C. 1. length
2. regexp_extract
3. upper
4. col("itemName")
5. as
D. 1. size
2. regexp_extract
3. lower
4. col("itemName")
5. alias
E. 1. lower
2. regexp_replace
3. length
4. "itemName"
5. alias

Answer: B

Explanation:
Explanation
Correct code block:
itemsDf.select(length(regexp_replace(lower(col("itemName")), "a|e|i|o|u|\s", "")).alias("consonant_ct")) Returned DataFrame:
+------------+
|consonant_ct|
+------------+
| 19|
| 16|
| 10|
+------------+
This question tries to make you think about the string functions Spark provides and in which order they should be applied. Arguably the most difficult part, the regular expression "a|e|i|o|u|
\s", is not a numbered blank. However, if you are not familiar with the string functions, it may be a good idea to review those before the exam.
The size operator and the length operator can easily be confused. size works on arrays, while length works on strings. Luckily, this is something you can read up about in the documentation.
The code block works by first converting all uppercase letters in column itemName into lowercase (the lower() part). Then, it replaces all vowels by "nothing" - an empty character "" (the regexp_replace() part). Now, only lowercase characters without spaces are included in the DataFrame. Then, per row, the length operator counts these remaining characters. Note that column itemName in itemsDf does not include any numbers or other characters, so we do not need to make any provisions for these. Finally, by using the alias() operator, we rename the resulting column to consonant_ct.
More info:
- lower: pyspark.sql.functions.lower - PySpark 3.1.2 documentation
- regexp_replace: pyspark.sql.functions.regexp_replace - PySpark 3.1.2 documentation
- length: pyspark.sql.functions.length - PySpark 3.1.2 documentation
- alias: pyspark.sql.Column.alias - PySpark 3.1.2 documentation
Static notebook | Dynamic notebook: See test 2

NEW QUESTION 66
The code block shown below should convert up to 5 rows in DataFrame transactionsDf that have the value 25 in column storeId into a Python list. Choose the answer that correctly fills the blanks in the code block to accomplish this.
Code block:
transactionsDf.__1__(__2__).__3__(__4__)

A. 1. filter
2. col("storeId")==25
3. take
4. 5
B. 1. filter
2. col("storeId")==25
3. toLocalIterator
4. 5
C. 1. filter
2. "storeId"==25
3. collect
4. 5
D. 1. select
2. storeId==25
3. head
4. 5
E. 1. filter
2. col("storeId")==25
3. collect
4. 5

Answer: A

Explanation:
Explanation
The correct code block is:
transactionsDf.filter(col("storeId")==25).take(5)
Any of the options with collect will not work because collect does not take any arguments, and in both cases the argument 5 is given.
The option with toLocalIterator will not work because the only argument to toLocalIterator is prefetchPartitions which is a boolean, so passing 5 here does not make sense.
The option using head will not work because the expression passed to select is not proper syntax. It would work if the expression would be col("storeId")==25.
Static notebook | Dynamic notebook: See test 1
(https://flrs.github.io/spark_practice_tests_code/#1/24.html ,
https://bit.ly/sparkpracticeexams_import_instructions)

NEW QUESTION 67
Which of the following code blocks returns a new DataFrame in which column attributes of DataFrame itemsDf is renamed to feature0 and column supplier to feature1?

A. itemsDf.withColumnRenamed(col("attributes"), col("feature0"), col("supplier"), col("feature1"))
B. itemsDf.withColumnRenamed(attributes, feature0).withColumnRenamed(supplier, feature1)
C. 1.itemsDf.withColumnRenamed("attributes", "feature0")
2.itemsDf.withColumnRenamed("supplier", "feature1")
D. itemsDf.withColumn("attributes", "feature0").withColumn("supplier", "feature1")
E. itemsDf.withColumnRenamed("attributes", "feature0").withColumnRenamed("supplier", "feature1")

Answer: E

Explanation:
Explanation
itemsDf.withColumnRenamed("attributes", "feature0").withColumnRenamed("supplier", "feature1") Correct! Spark's DataFrame.withColumnRenamed syntax makes it relatively easy to change the name of a column.
itemsDf.withColumnRenamed(attributes, feature0).withColumnRenamed(supplier, feature1) Incorrect. In this code block, the Python interpreter will try to use attributes and the other column names as variables. Needless to say, they are undefined, and as a result the block will not run.
itemsDf.withColumnRenamed(col("attributes"), col("feature0"), col("supplier"), col("feature1")) Wrong. The DataFrame.withColumnRenamed() operator takes exactly two string arguments. So, in this answer both using col() and using four arguments is wrong.
itemsDf.withColumnRenamed("attributes", "feature0")
itemsDf.withColumnRenamed("supplier", "feature1")
No. In this answer, the returned DataFrame will only have column supplier be renamed, since the result of the first line is not written back to itemsDf.
itemsDf.withColumn("attributes", "feature0").withColumn("supplier", "feature1") Incorrect. While withColumn works for adding and naming new columns, you cannot use it to rename existing columns.
More info: pyspark.sql.DataFrame.withColumnRenamed - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 3

NEW QUESTION 68
The code block shown below should add column transactionDateForm to DataFrame transactionsDf. The column should express the unix-format timestamps in column transactionDate as string type like Apr 26 (Sunday). Choose the answer that correctly fills the blanks in the code block to accomplish this.
transactionsDf.__1__(__2__, from_unixtime(__3__, __4__))

A. 1. withColumn
2. "transactionDateForm"
3. "transactionDate"
4. "MM d (EEE)"
B. 1. withColumn
2. "transactionDateForm"
3. "MMM d (EEEE)"
4. "transactionDate"
C. 1. select
2. "transactionDate"
3. "transactionDateForm"
4. "MMM d (EEEE)"
D. 1. withColumnRenamed
2. "transactionDate"
3. "transactionDateForm"
4. "MM d (EEE)"
E. 1. withColumn
2. "transactionDateForm"
3. "transactionDate"
4. "MMM d (EEEE)"

Answer: E

Explanation:
Explanation
Correct code block:
transactionsDf.withColumn("transactionDateForm", from_unixtime("transactionDate", "MMM d (EEEE)")) The question specifically asks about "adding" a column. In the context of all presented answers, DataFrame.withColumn() is the correct command for this. In theory, DataFrame.select() could also be used for this purpose, if all existing columns are selected and a new one is added.
DataFrame.withColumnRenamed() is not the appropriate command, since it can only rename existing columns, but cannot add a new column or change the value of a column.
Once DataFrame.withColumn() is chosen, you can read in the documentation (see below) that the first input argument to the method should be the column name of the new column.
The final difficulty is the date format. The question indicates that the date format Apr 26 (Sunday) is desired. The answers give "MMM d (EEEE)" and "MM d (EEE)" as options. It can be hard to know the details of the date format that is used in Spark. Specifically, knowing the differences between MMM and MM is probably not something you deal with every day. But, there is an easy way to remember the difference: M (one letter) is usually the shortest form: 4 for April. MM includes padding: 04 for April. MMM (three letters) is the three-letter month abbreviation: Apr for April. And MMMM is the longest possible form: April. Knowing this four-letter sequence helps you select the correct option here.
More info: pyspark.sql.DataFrame.withColumn - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 3

NEW QUESTION 69
The code block displayed below contains an error. The code block should return a copy of DataFrame transactionsDf where the name of column transactionId has been changed to transactionNumber. Find the error.
Code block:
transactionsDf.withColumn("transactionNumber", "transactionId")

A. The copy() operator should be appended to the code block to ensure a copy is returned.
B. Each column name needs to be wrapped in the col() method and method withColumn should be replaced by method withColumnRenamed.
C. The arguments to the withColumn method need to be reordered and the copy() operator should be appended to the code block to ensure a copy is returned.
D. The arguments to the withColumn method need to be reordered.
E. The method withColumn should be replaced by method withColumnRenamed and the arguments to the method need to be reordered.

Answer: E

Explanation:
Explanation
Correct code block:
transactionsDf.withColumnRenamed("transactionId", "transactionNumber")
Note that in Spark, a copy is returned by default. So, there is no need to append copy() to the code block.
More info: pyspark.sql.DataFrame.withColumnRenamed - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 2

NEW QUESTION 70
The code block displayed below contains an error. The code block should combine data from DataFrames itemsDf and transactionsDf, showing all rows of DataFrame itemsDf that have a matching value in column itemId with a value in column transactionsId of DataFrame transactionsDf. Find the error.
Code block:
itemsDf.join(itemsDf.itemId==transactionsDf.transactionId)

A. The join expression is malformed.
B. The join method is inappropriate.
C. The union method should be used instead of join.
D. The join statement is incomplete.
E. The merge method should be used instead of join.

Answer: D

Explanation:
Explanation
Correct code block:
itemsDf.join(transactionsDf, itemsDf.itemId==transactionsDf.transactionId) The join statement is incomplete.
Correct! If you look at the documentation of DataFrame.join() (linked below), you see that the very first argument of join should be the DataFrame that should be joined with. This first argument is missing in the code block.
The join method is inappropriate.
No. By default, DataFrame.join() uses an inner join. This method is appropriate for the scenario described in the question.
The join expression is malformed.
Incorrect. The join expression itemsDf.itemId==transactionsDf.transactionId is correct syntax.
The merge method should be used instead of join.
False. There is no DataFrame.merge() method in PySpark.
The union method should be used instead of join.
Wrong. DataFrame.union() merges rows, but not columns as requested in the question.
More info: pyspark.sql.DataFrame.join - PySpark 3.1.2 documentation, pyspark.sql.DataFrame.union - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 3

NEW QUESTION 71
Which of the following code blocks returns all unique values of column storeId in DataFrame transactionsDf?

A. transactionsDf.select("storeId").distinct()
(Correct)
B. transactionsDf.filter("storeId").distinct()
C. transactionsDf["storeId"].distinct()
D. transactionsDf.select(col("storeId").distinct())
E. transactionsDf.distinct("storeId")

Answer: A

Explanation:
Explanation
distinct() is a method of a DataFrame. Knowing this, or recognizing this from the documentation, is the key to solving this question.
More info: pyspark.sql.DataFrame.distinct - PySpark 3.1.2 documentation Static notebook | Dynamic notebook: See test 2

NEW QUESTION 72
The code block shown below should set the number of partitions that Spark uses when shuffling data for joins or aggregations to 100. Choose the answer that correctly fills the blanks in the code block to accomplish this.
spark.sql.shuffle.partitions
__1__.__2__.__3__(__4__, 100)

A. 1. pyspark
2. config
3. set
4. "spark.sql.shuffle.partitions"
B. 1. pyspark
2. config
3. set
4. spark.shuffle.partitions
C. 1. spark
2. conf
3. set
4. "spark.sql.aggregate.partitions"
D. 1. spark
2. conf
3. set
4. "spark.sql.shuffle.partitions"
E. 1. spark
2. conf
3. get
4. "spark.sql.shuffle.partitions"

Answer: D

Explanation:
Explanation
Correct code block:
spark.conf.set("spark.sql.shuffle.partitions", 100)
The conf interface is part of the SparkSession, so you need to call it through spark and not pyspark. To configure spark, you need to use the set method, not the get method. get reads a property, but does not write it. The correct property to achieve what is outlined in the question is spark.sql.aggregate.partitions, which needs to be passed to set as a string. Properties spark.shuffle.partitions and spark.sql.aggregate.partitions do not exist in Spark.
Static notebook | Dynamic notebook: See test 2

NEW QUESTION 73
Which of the following code blocks returns a one-column DataFrame for which every row contains an array of all integer numbers from 0 up to and including the number given in column predError of DataFrame transactionsDf, and null if predError is null?
Sample of DataFrame transactionsDf:
1.+-------------+---------+-----+-------+---------+----+
2.|transactionId|predError|value|storeId|productId| f|
3.+-------------+---------+-----+-------+---------+----+
4.| 1| 3| 4| 25| 1|null|
5.| 2| 6| 7| 2| 2|null|
6.| 3| 3| null| 25| 3|null|
7.| 4| null| null| 3| 2|null|
8.| 5| null| null| null| 2|null|
9.| 6| 3| 2| 25| 2|null|
10.+-------------+---------+-----+-------+---------+----+

A. 1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = list(range(target))
6. return result
7.
8.count_to_target_udf = udf(count_to_target)
9.
10.transactionsDf.select(count_to_target_udf('predError'))
B. 1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = [range(target)]
6. return result
7.
8.count_to_target_udf = udf(count_to_target, ArrayType[IntegerType])
9.
10.transactionsDf.select(count_to_target_udf(col('predError')))
C. 1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = list(range(target))
6. return result
7.
8.count_to_target_udf = udf(count_to_target, ArrayType(IntegerType()))
9.
10.transactionsDf.select(count_to_target_udf('predError'))
(Correct)
D. 1.def count_to_target(target):
2. result = list(range(target))
3. return result
4.
5.count_to_target_udf = udf(count_to_target, ArrayType(IntegerType()))
6.
7.df = transactionsDf.select(count_to_target_udf('predError'))
E. 1.def count_to_target(target):
2. if target is None:
3. return
4.
5. result = list(range(target))
6. return result
7.
8.transactionsDf.select(count_to_target(col('predError')))

Answer: C

Explanation:
Explanation
Correct code block:
def count_to_target(target):
if target is None:
return
result = list(range(target))
return result
count_to_target_udf = udf(count_to_target, ArrayType(IntegerType()))
transactionsDf.select(count_to_target_udf('predError'))
Output of correct code block:
+--------------------------+
|count_to_target(predError)|
+--------------------------+
| [0, 1, 2]|
| [0, 1, 2, 3, 4, 5]|
| [0, 1, 2]|
| null|
| null|
| [0, 1, 2]|
+--------------------------+
This question is not exactly easy. You need to be familiar with the syntax around UDFs (user-defined functions). Specifically, in this question it is important to pass the correct types to the udf method - returning an array of a specific type rather than just a single type means you need to think harder about type implications than usual.
Remember that in Spark, you always pass types in an instantiated way like ArrayType(IntegerType()), not like ArrayType(IntegerType). The parentheses () are the key here - make sure you do not forget those.
You should also pay attention that you actually pass the UDF count_to_target_udf, and not the Python method count_to_target to the select() operator.
Finally, null values are always a tricky case with UDFs. So, take care that the code can handle them correctly.
More info: How to Turn Python Functions into PySpark Functions (UDF) - Chang Hsin Lee - Committing my thoughts to words.
Static notebook | Dynamic notebook: See test 3

NEW QUESTION 74
Which of the following statements about RDDs is incorrect?

A. An RDD consists of a single partition.
B. The high-level DataFrame API is built on top of the low-level RDD API.
C. RDD stands for Resilient Distributed Dataset.
D. RDDs are great for precisely instructing Spark on how to do a query.
E. RDDs are immutable.

Answer: A

Explanation:
Explanation
An RDD consists of a single partition.
Quite the opposite: Spark partitions RDDs and distributes the partitions across multiple nodes.

NEW QUESTION 75
Which of the following code blocks returns the number of unique values in column storeId of DataFrame transactionsDf?

A. transactionsDf.select(count("storeId")).dropDuplicates()
B. transactionsDf.select("storeId").dropDuplicates().count()
C. transactionsDf.dropDuplicates().agg(count("storeId"))
D. transactionsDf.select(distinct("storeId")).count()
E. transactionsDf.distinct().select("storeId").count()

Answer: B

Explanation:
Explanation
transactionsDf.select("storeId").dropDuplicates().count()
Correct! After dropping all duplicates from column storeId, the remaining rows get counted, representing the number of unique values in the column.
transactionsDf.select(count("storeId")).dropDuplicates()
No. transactionsDf.select(count("storeId")) just returns a single-row DataFrame showing the number of non-null rows. dropDuplicates() does not have any effect in this context.
transactionsDf.dropDuplicates().agg(count("storeId"))
Incorrect. While transactionsDf.dropDuplicates() removes duplicate rows from transactionsDf, it does not do so taking only column storeId into consideration, but eliminates full row duplicates instead.
transactionsDf.distinct().select("storeId").count()
Wrong. transactionsDf.distinct() identifies unique rows across all columns, but not only unique rows with respect to column storeId. This may leave duplicate values in the column, making the count not represent the number of unique values in that column.
transactionsDf.select(distinct("storeId")).count()
False. There is no distinct method in pyspark.sql.functions.

NEW QUESTION 76
Which of the following code blocks selects all rows from DataFrame transactionsDf in which column productId is zero or smaller or equal to 3?

A. transactionsDf.where("productId"=3).or("productId"<1))
B. transactionsDf.filter(productId==3 or productId<1)
C. transactionsDf.filter((col("productId")==3) or (col("productId")<1))
D. transactionsDf.filter(col("productId")==3 | col("productId")<1)
E. transactionsDf.filter((col("productId")==3) | (col("productId")<1))

Answer: E

Explanation:
Explanation
This question targets your knowledge about how to chain filtering conditions. Each filtering condition should be in parentheses. The correct operator for "or" is the pipe character (|) and not the word or. Another operator of concern is the equality operator. For the purpose of comparison, equality is expressed as two equal signs (==).
Static notebook | Dynamic notebook: See test 2

NEW QUESTION 77
......

Free Associate-Developer-Apache-Spark Dumps are Available for Instant Access: https://www.passsureexam.com/Associate-Developer-Apache-Spark-pass4sure-exam-dumps.html

Based on Official Syllabus Topics of Actual Databricks Associate-Developer-Apache-Spark Exam: https://drive.google.com/open?id=1x-3QMTKPpPIQhLCGtPariJdIdoM2Wz2h

2022 Current Associate-Developer-Apache-Spark dumps Preparation through Our Practice Test [Q52-Q77]

The best way to study for a Databricks Associate Developer Apache Spark Exam is by getting as many

Understand why Databricks experts work only for top consulting companies

For more info read the reference:

Related Articles

Useful Links

Latest Exam Braindumps

Contact Us