All Questions
Tagged with janusgraph apache-spark
31
questions
1
vote
1
answer
61
views
How to read janusgraph vertexid from Cassandra db
I have a janusgraph database that uses Cassandra and ElasticSearch.
The database is huge and I need a way to traverse through all the vertexes of the janusgraph for specific use case.
I have tried ...
- 348
1
vote
1
answer
352
views
Gremlin query on Janusgraph through Spark. Error: Provider org.janusgraph.hadoop.serialize.JanusGraphKryoShimService could not be instantiated
Current Architecture
Description
I am using JanusGraph 0.6.2 for graph processing.
GCP BigTable as JanusGraph Backend/database.
Spark 3.0.0 with hadoop 2.7 for data processing, setup locally (...
- 79
0
votes
1
answer
165
views
java.io.InvalidClassException: org.apache.tinkerpop.gremlin.hadoop.structure.HadoopConfiguration; unable to create instance
I am totally new to Spark, My spark master is on GCP Kubernetes Standard cluster.
I am using a gremlin client to trigger the spark job, on my local machine configuration it works totally fine and the ...
- 79
1
vote
0
answers
92
views
Reading TinkerPop library generated files using Spark
Is there a direct way of reading tinkerpop format org.apache.tinkerpop.gremlin.hadoop.structure.io.ObjectWritable files using spark.
Spark version: 3.*
- 91
1
vote
0
answers
316
views
Gremlin console and spark UI not responding when performing OLAP query with JanusGraph with Apache spark
I have a graph on Janusgraph(v0.5.3) which contains around 2 million vertices and 20 million edges. I'm making a OLAP query which is modified version of lowest_common_ancestor recipe (query added ...
- 1,866
0
votes
1
answer
677
views
Setting up JanusGraph
I'm new to JanusGraph. Can somebody help me edit this docker-compose file to use ScyllaDB instead of Cassandra and Apache Solr instead of Elasticsearch? Also does Apache Spark get installed ...
- 151
0
votes
1
answer
573
views
Gremlin Spark Java Maven Project - Slow query response
I have wrote a program in order to perform some queries on top of Gremlin (I use Janus Graph with Cassandra and Solr as the engine) with the help of Spark, but the query result is terrible slow.
Most ...
6
votes
2
answers
1k
views
How Spark can speed up bulk loading to JanusGraph?
I need to load lots of vertices and edges to JanusGraph with Cassandra backend from other storage. I've read about bulk loading and Spark configuring (https://docs.janusgraph.org/advanced-topics/bulk-...
- 85
-1
votes
1
answer
924
views
How to solve java.lang.ClassNotFoundException: org.apache.tinkerpop.gremlin.spark.structure.io.gryo.GryoSerializer
I am using tinkerpop + Janus Graph + Spark
build.gradle
compile group: 'org.apache.tinkerpop', name: 'spark-gremlin', version: '3.1.0-incubating'
below is some critical configuration that we have
...
- 8,979
0
votes
0
answers
89
views
janus graph vertex multivalue becomes single value when control enter in reduce phase , can any one suggest on this?
we are using tinkerpop vertex program custom implemented and implemented map-reduce custom implementation , we have data in Janus graph with for example sample vertex with different properties, with ...
- 8,979
1
vote
1
answer
214
views
Creating spark cluster with drone.yml not working
I have docker-compose.yml with below image and configuration
version: '3'
services:
spark-master:
image: bde2020/spark-master:2.4.4-hadoop2.7
container_name: spark-master
ports:
- ...
- 8,979
3
votes
1
answer
401
views
Error committing 10k records to Janus graph with cassandra
I'm fetching around 10 million records from a oracle DB and trying to persist those to Janus graph with Cassandra as storage backend [using Spark framework].
When i tried iterating the records in a ...
- 31
1
vote
1
answer
2k
views
Push Spark Dataframe to Janusgraph for Spark running in EMR
I have an Janusgraph running on EC2 Instance and for its backend storage i have Cassandra cluster on EC2 as well. I want to push the aggregated and filter data to Janusgraph from python code Apache ...
- 20.4k
1
vote
1
answer
549
views
Janusgraph OLAP query outside gremlin console
I have a graph in which some nodes are having millions of incoming edges. I need to obtain the edge count of such nodes periodically. I'm using cassandar as storage backend.
Query :
g.V().has('vid','...
- 143
3
votes
0
answers
593
views
how is JanusGraph doing with Spark, are there any latest examples experience with it?
has anyone experienced with spark streaming writing and reading to JanusGraph? I am trying following code from tinkerpop tests
<dependency>
<groupId>org.apache.tinkerpop&...
- 220