प्रौद्योगिकी साझेदारी

한어Русский языкEnglishFrançaisIndonesianSanskrit日本語DeutschPortuguêsΕλληνικάespañolItalianoSuomalainenLatina

spark sql इत्यस्मिन् उपयोक्तारः Join hint इत्यस्य उपयोगं कृत्वा सूचयितुं शक्नुवन्ति यत् Join Spark इत्यस्य कस्य प्रकारस्य उपयोगः कर्तव्यः इति । Spark 3.0 इत्यस्मात् पूर्वं केवलं BROADCAST इत्यादीनि Join संकेतानि समर्थितानि आसन् । Spark 3.0 तः आरभ्य MERGE, SHUFFLE_HASH तथा SHUFFLE_REPLICATE_NL इति त्रयः Join Hints योजिताः सन्ति । प्राथमिकता BROADCAST > MERGE > SHUFFLE_HASH > SHUFFLE_REPLICATE_NL अस्ति । यदि Join इत्यस्य उभयतः BROADCAST अथवा SHUFFLE_HASH योजितं भवति तर्हि Spark joinType इत्यस्य आधारेण उभयपक्षस्य आकारस्य च आधारेण कः पक्षः निर्मातव्यः इति चिनोति ।

-- Join Hints for broadcast join
SELECT /*+ BROADCAST(t1) */ * FROM t1 INNER JOIN t2 ON t1.key = t2.key;
SELECT /*+ BROADCASTJOIN (t1) */ * FROM t1 left JOIN t2 ON t1.key = t2.key;
SELECT /*+ MAPJOIN(t2) */ * FROM t1 right JOIN t2 ON t1.key = t2.key;
 
-- Join Hints for shuffle sort merge join
SELECT /*+ SHUFFLE_MERGE(t1) */ * FROM t1 INNER JOIN t2 ON t1.key = t2.key;
SELECT /*+ MERGEJOIN(t2) */ * FROM t1 INNER JOIN t2 ON t1.key = t2.key;
SELECT /*+ MERGE(t1) */ * FROM t1 INNER JOIN t2 ON t1.key = t2.key;
 
-- Join Hints for shuffle hash join
SELECT /*+ SHUFFLE_HASH(t1) */ * FROM t1 INNER JOIN t2 ON t1.key = t2.key;
 
-- Join Hints for shuffle-and-replicate nested loop join
SELECT /*+ SHUFFLE_REPLICATE_NL(t1) */ * FROM t1 INNER JOIN t2 ON t1.key = t2.key;
 
-- When different join strategy hints are specified on both sides of a join, Spark
-- prioritizes the BROADCAST hint over the MERGE hint over the SHUFFLE_HASH hint
-- over the SHUFFLE_REPLICATE_NL hint.
-- Spark will issue Warning in the following example
-- org.apache.spark.sql.catalyst.analysis.HintErrorLogger: Hint (strategy=merge)
-- is overridden by another hint and will not take effect.
SELECT /*+ BROADCAST(t1), MERGE(t1, t2) */ * FROM t1 INNER JOIN t2 ON t1.key = t2.key;

स्पार्क संकेते उपयोगसम्बन्धः: https://blog.51cto.com/u_15435003/5296344