Run a Spark Scala/Java application on an HDInsight clusterĬreate a spark Scala/Java application, then run the application on a Spark cluster by doing the following steps:Ĭlick Add Configuration to open Run/Debug Configurations window. This example uses the AgeMean_Div() script. Select src > main > scala to open your code in the project. In the Spark Version drop-down list, select Spark 2.3.2(Scala 2.11.8). In the Project SDK drop-down list, select Java 1.8 for Spark 2.3.2 cluster.Ĭ. Enter a project name and project location.ī. In the New Project window, do the following steps:Ī. Select Spark Project with Failure Task Debugging Sample(Preview)(Scala) from the main window. Select Azure Spark/HDInsight from the left pane.ī. See Download Microsoft Azure Storage Explorer.Ĭreate a spark2.3.2 project to continue failure debug, take failure task debugging sample file in this document. See Installing the Azure Toolkit for IntelliJ.Ĭonnect to your HDInsight cluster. This article uses IntelliJ IDEA Community 2019.1.3.Īzure Toolkit for IntelliJ. It's good that we have it.This article provides step-by-step guidance on how to use HDInsight Tools in Azure Toolkit for IntelliJ to run Spark Failure Debug applications. Usually, if I start the debugger for other programming languages like Java or Go it's working right from the beginning - sometimes without even any configuration needed at all.ĭisclaimer: This is just me speaking about my experience. Maybe it's me, maybe my process of thinking is just not compatible with the way Xdebug is working.But I know I'm not alone. I'd say I'm a quite seniorish developer and I work with PHP for like 20 years - but until today Xdebug and me never developed a friendship. Is it the server XDebug configuration? Is it some port mapping issue? Something with Docker/Vagrant/VirtualBox? Is something wrongly configured in the IDE? Hard to find out if you don't have any hints. Additionally, if it's not working, you don't immediately know where to start searching for the issue. For sure we missed something, but the amount of different configuration options makes it hard to set up in my eyes. Even for the same project - if it was working for me - it wasn't for a collegaue - even if we compared settings. If I want to set it up for a new project I almost always struggle - what means I have to google around, compare configs, try this and that in my IDE and php.ini until it finally works and I don't exactly know why.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |