Questions tagged [azureml-python-sdk]
The azureml-python-sdk tag has no usage guidance.
azureml-python-sdk
217
questions
0
votes
1
answer
67
views
Load Registered Component in Azure ML for Pipeline using Python sdk v2
I'm working in Azure Machine Learning Studio to create components that I will run together in a pipeline. In this basic example, I have a single python script and a single yml file that make up my ...
1
vote
0
answers
54
views
Azureml studio Unable to start connection test getting ValidationError: Required metadata property ContainerName is missing
In Azureml studio i go into service connection and try to test the connection using Microsoft Entra but getting " Required metadata property ContainerName is missing."
however when i go ...
0
votes
0
answers
48
views
AzureML SDKV2 - ml_client job stream error - Unauthorized
I have a aml pipeline that uses azureml sdkv2. I submit the pipeline as:
def get_chained_credentials(client_id: str):
try:
managed_identity_creds = ManagedIdentityCredential(client_id=...
- 750
0
votes
1
answer
84
views
How to set resources in pipeline component in Azure machine learning sdk v2?
I'm creating a pipeline that scores images using a custom pytorch unet. Assume I registered my model and now I'm creating that pipeline. I'm running into a OOM issue due to Azure setting the docker ...
- 118
0
votes
1
answer
98
views
AzureML command job: 'Your file exceeds 100 MB.'
I am trying to submit a job to AzureML using Python SDK V2.
I am developing locally on VSCode.
As my repository grew over time, I now get the message when submitting the job:
Your file exceeds 100 MB....
- 15
1
vote
0
answers
164
views
Connection Default_AzureOpenAI not found Azure AI Studio
**Update: It's a shame Microsoft set a redirect of the tutorial I was using for this Stackoverflow question and the tutorial is now lost and replaced without it being the same tutorial or code. If ...
- 640
0
votes
1
answer
201
views
Azure deployment error "TypeError: metaclass conflict" when importing tensorflow (1.13.1) in project code
Any help fixing the problem would be greatly appreciated.
I am trying to deploy an old CNN model (MRCNN), last deployed in December 2020, on Azure ML studio using the Python SDK V2. After a successful ...
- 2,066
0
votes
1
answer
107
views
error when trying to create a scheduled job in azure ml
im getting this error when submitting a scheduled pipeline,
Failed to submit job due to Exception: Response status code does not indicate success: 400 (Exception(s) encountered while validating ...
0
votes
1
answer
126
views
Azure Machine Learning dataset creation hangs forever
I'm trying to create a Dataset from a datastore using Azure ML, however, the execution hangs forever and never finishes.
This is the code I'm running which I've adapted from the Msft documentation:
...
- 14.6k
0
votes
1
answer
329
views
Azure ML custom command component - how to use a custom image
All examples that I have seen of the use of the command_component decorator to create custom components in Azure ML pipelines use an image from Microsoft's container registry, as seen in this ...
- 338
0
votes
2
answers
349
views
Unable to read current version of delta table in azure ml studio using Data asset
I am trying to create data assest with ADLS gen 2, and read a delta table on adls gen folder something like this:
/
└── my-data
├── _delta_log
├── part-0000-xxx.parquet
└── part-0001-xxx....
- 1
1
vote
1
answer
132
views
Read shapefile from Azure Blob Storage in Azure Machine Learning
I have several shapefiles stored in Azure Blob Storage within my Azure Machine Learning workspace, each comprising the files: file.fix, file.shp, file.dbf, file.prj, and file.shx. I need to directly ...
- 37
0
votes
1
answer
209
views
Azure ML Pipeline: Specify input-path on compute
I got a python script which shall be executed as an Azure ML pipeline step. This script expects to find several file-sets in a certain tree structure e.g.
data/
├─ project_A/
│ ├─ data.csv
│ ├─ ...
- 50
0
votes
1
answer
90
views
What is the best way to containarise model using Azure ML Pipeline
We are migrating from Azure ML Python SDK V1 to V2.
I have a requirement to train a model, containerize using a custom Dockerfile, and push it to a container registry.
Previously with V1 SDK, I used ...
- 835
0
votes
1
answer
177
views
How do I write a pandas data frame from an Azure ML Jupyter notebook to an Azure SQL database?
I'm working with some data in Azure ML, using Jupyter notebooks.
I've gathered the data from an Azure SQL database by setting the tables up as assets and then used Python to run some machine learning ...
- 11
4
votes
1
answer
222
views
AzureML Batch endpoint invocation: Failed to load dataset definition with azureml-dataprep==4.12.9
Context
Images are uploaded to a folder in Blob storage. An AzureML Data asset is created pointing to that folder, and it is used as data input when invoking a batch inference endpoint in AzureML.
The ...
- 658
0
votes
0
answers
97
views
getting mlflow exception in azure ml compute cluster if I am trying to get the latest version from production
I was trying to get a latest model from production using mlflow command,
version = client.get_latest_versions(model_name, stages=["Production"])[0].version
However this is working fine in ...
-1
votes
1
answer
114
views
Update the content of a Dataset in AzureML studio
In the execution of a job in my MLOps pipeline, I want to obtain a dataset's content, modify it, and upload the new content, overwriting the old content of the file. Is there any option to do it? I ...
0
votes
0
answers
164
views
Azure ML: how to use remote location as code path?
I am using Azure ML python SDK v2 to submit the job. The code parameter at "command" is pointed to the local "./source" path.
train_func = command(
environment="AzureML-...
- 1
0
votes
0
answers
177
views
How to deploy different endpoints of different models to a single virtual machine in azure ml
I have 6 different models trained. I registered models and created endpoints separately in azure ml and able to access endpoints and getting outputs. But noticed that each endpoint took one virtual ...
- 11
1
vote
1
answer
207
views
How to deploy two dependent models in azure ml
I have two YOLO models. In the processing script, the output of model 1 is given as input to model 2 and model 2 gives final output after further more processing. My input to model 1 is either jpg or ...
- 11
4
votes
1
answer
309
views
Azure ML: DeploymentIdentityError: Failed to create Kubernetes deployment identity, Reason:RefreshExtensionIdentityNotSet
I'm using Azure Machine Learning v2 SDK to create a model deployment on a kubernetes compute attached to an AML workspace. I'm able to deploy it locally as part of testing before deploying online. ...
0
votes
0
answers
312
views
Azure ML Cluster: changing folders during job
I am working with Azure ML to train and test an ML model. I have a main script which calls different functions (.py files in different subfolders) to load data, preprocess, train and test model. While ...
0
votes
0
answers
57
views
StreamError sharing inputs form Sweep Job to Command Job - AzureML SDKv2
We have AML pipelines (sdk v2) containing three steps i.e training, image_creation and model_selector. We are using command components for these three steps.
All of our steps can run in isolation as ...
- 750
0
votes
0
answers
275
views
Azure Machine Learning SDK V2 - Sequencing pipeline steps with sweep job
We are trying to migrate our azure ml pipelines from SDKV1 to SDKV2. The problem we are facing is that we cannot sequence the pipeline steps like we used to to in SDKV1.
In SDKV1, we could simply do ...
- 750
0
votes
1
answer
213
views
can you load ml models from azureml registry to an Azure function app
Is it possible to load registered ml models from azureml model registry to an azure function app and carry out inference or do these models need to be downloaded inside the function app first? any ...
user12704077
0
votes
1
answer
1k
views
Register dataset in AzureML Python SDK v2 from a dataframe
I am looking to upgrade to the new Azure AML Python SDK v2.
However, I can't replicate registering a dataset (now called data assets) directly from a dataframe. it seems that now the only option is to ...
- 37
0
votes
0
answers
232
views
Issue with Azure ML job command
I am trying to run a job in Azure Ml using job command on a compute cluster. I have stored the data in a folder in Azure ML. Everytime I create and run the job command it displays the following ...
0
votes
0
answers
133
views
Download particular source file from AzureML snapshot directly
The Azure Machine Learning SDK for Python provides the restore_snapshot() method to download a zip file with the source code of an AzureML run. Assume that I am interested in inspecting a certain ...
- 8,242
0
votes
1
answer
338
views
Azure container registry detail using Azure SDK for Python
As we know that to create a environment in Azure Machine Learning Workspace we need Conda dependencies file and base image. In my case you can see.
Build process using this conda file and base image ...
- 2,185
0
votes
1
answer
431
views
how to load a registered model in azureml to a azure function app?
I am migrating to Azure function app for model inference from using Azure container instance deployment. The code below shows how I was loading the model in score.py
import json
import pandas as pd
...
user14736700
0
votes
0
answers
113
views
Azure ML Pipelines user_identity on parallel job
I'm using a user_identity to read data from Azure Data Lake. I'm saving the data to a datastore. Then I want to use the datastore for my parallel job, but I keep running into this error:
Please ...
0
votes
1
answer
981
views
Azure Machine Learning SDK V1 migration to V2 Pipeline Steps
I am trying to migrate Pipelines from Azure Machine Learning SDK V1 to V2, but sometimes I don't understand the logic behind the V2 and I get stuck.
In V1, I just had to create PythonScriptStep and ...
- 859
1
vote
1
answer
1k
views
How to download a data asset from Azure registry
As the question in title
I can use azure.ai.ml.MLClient to connect with an azure registry.
In this registry, there is a data asset (container) named as train, and there are a few jsonl files saved in ...
- 1,117
0
votes
2
answers
532
views
unable to import azureml in vscode
I've installed python 3.8, from vscode terminal I created an env and seem to have successfully installed azureml as below:
C:\Python38\python.exe -m venv tempml
tempml\scripts\activate
pip install ...
- 469
0
votes
0
answers
113
views
Cancel run in azureml sdk2
I'm using azureml sdk2 and I want to add execution cancel condition to my code but I don't know how to do it?
from azure.ai.ml import MLClient
ml_client = MLClient(
credential=credential,
...
- 133
0
votes
1
answer
362
views
build and deploy docker container to build fast api app on Azure web app
I am currently testing a small project and developing within the Azure ML notebook environemnt. Due to the confidential issue, I have to test within the Azure. I have a python script that I would like ...
- 345
1
vote
1
answer
492
views
Azure ml Studio Object Detection jupyter: Error to access the mltable for training the model
I am new with azure machine learning and I am still learnning. There is a file named annotations.jsonl saved in the subfolder "image-frames/dados_treinamento" inside the container "...
- 107
0
votes
0
answers
147
views
Use Azure ML Environment from Share ML Registry with Python SDK
We have created an environment in azure ML shared registry. The environment was built into the registry. Here is what it looks like:
A DevOps pipeline creates this environment:
az ml environment ...
- 750
0
votes
1
answer
322
views
How do you dynamically change the asset input in a pipeline?
In AzureML, I have a pipeline that takes in a data asset. The pipeline is now published and I'd like to change that data asset. How do you do that? There don't seem to be docs for this, but from the ...
- 3,868
0
votes
1
answer
4k
views
How to load pdf files from Azure Blob Storage with LangChain PyPDFLoader
I currently trying to implement langchain functionality to talk with pdf documents.
I have a bunch of pdf files stored in Azure Blob Storage. I am trying to use langchain PyPDFLoader to load the pdf ...
- 345
0
votes
0
answers
141
views
AzureML SDK v2 list compute error AttributeError: 'Databricks' object has no attribute 'tags'
I am using AzureML SDK v2. I run the following code
ml_ws_client = MLClient(credential=credential,subscription_id=subscription_id, resource_group_name=resource_group, workspace_name=ws_name)
for ...
- 181
1
vote
0
answers
233
views
HTTP Error 409 while accessing registered dataset as a URI file (Azure SDKv2)
In Azure ML I'm trying to read data from the blob storage, register it as a URI File. This is needed to keep data versioning and at the same time to keep data lineage (sort of) because components ...
- 333
3
votes
3
answers
2k
views
How to set environment variables in pipelines in Azure ML SDK v2 with jobs.create_or_update()?
I am changing some of our code from Azure ML's SDK v1 to v2.
However, when I invoke pipelines with components via ml_client.jobs.create_or_update, I just can't get them to use my environment variables....
- 113
0
votes
1
answer
176
views
How to modify azureml python sdk v2 to serve custom use cases?
I want to modify azureml.data.dataset_factory.register_pandas_dataframe() for my use case so it returns relative_path_with_guid in addition to the registered_dataset by default.
The default azureml....
0
votes
1
answer
523
views
How to trigger an event-based based pipeline in AzureML?
I have a published pipeline in AzureML that preprocess the data and train a new model. I am trying an event-based schedule so that whenever a new dataset is registered in the workspace, it triggers ...
- 3
4
votes
0
answers
1k
views
Is there a way to create virtual env (and kernel) in Azure ML notebook from a custom registered environment?
I have registered a custom environment (through Environments tab) in Azure Machine Learning (providing Docker file and conda_dependencies.yaml) which I plan to use for training jobs. Now I would like ...
1
vote
2
answers
929
views
Error on Azure ML - AutoMLPipelineBuilder
I want to build a Pipeline in Azure ML. The Training pipeline runs well.
Training:
training_pipeline_steps = AutoMLPipelineBuilder.get_many_models_train_steps(
experiment=experiment,
...
- 41
3
votes
0
answers
429
views
Authentication unavailable when i load data_asset with MLTABLE in Azure ML SDK Python2
credential = DeviceCodeCredential()
ml_client = MLClient(credential,
auth_subscription_id,
auth_resource_group,
...
- 111
0
votes
1
answer
494
views
MLmodel local deployment with azure python sdk
I'm trying to deploy a mlflow model locally using azure sdk for python. I'm following this example https://github.com/Azure/azureml-examples/blob/main/sdk/python/endpoints/online/mlflow/online-...
- 594