Cloud - Read from Local Network File Storage (e.g. via Azure Function)

Question

I am looking to use python code deployed in Azure Cloud (e.g. on an Azure Function), to read files location from a shared/local network drive (on-premise). The network drive requires user based authentication (i.e. different users have access/permissions to different areas).

There is an express route connection established to the local network, and I also have confirmed looking at the firewall there is nothing blocking cloud access to the shared drives.

My question is - what is the best way to read these files from local network (on-premise) to within the Azure function using python? E.g. what packages, should be used and how can user credentials be entered also.

I am unable to access with smbprotocol and can't see the files (e.g. using the os package), and believe its due to authentication.

Answer 1

You can achieve the task with python, and now that we know the connection requires authentication maybe we need to know more about the express route connection, the Azure Function (python) could require server-side and client-side connectors, however if we need to keep the current security, it is always a good idea to keep the same cloud infrastructure, we can implement a type of customer premises equipment (CPE) network connection (2 months ago this feature was still in preview):

Client or local python:

pip install smbprotocol #windows or #python3 -m pip install smbprotocol

Linux will require Kerberos support: pip install smbprotocol[kerberos]

Once Client or local python has already authenticated the access to the Network Drive, you can manage it as a normal local file to read, and then we should move to the server side (Azure Function), you need to create an Azure Function to be allowed to post a file:

It will be a combination of: Creating your Azure Function to receive a FILE with a POST function and store it on Azure Blob or similar, or process it in the way desired (https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-decorators#sdk-type-bindings-examples)

The function I would use for blob, client side:

def s_on_db(self,container_db,folder_db,file_db_local_complete_path,file_to_file_to_db):
    try:
        connection_string="BlobEndpoint=https://mystorageaccount2024.blob.core.windows.net/;QueueEndpoint=h..."
        blob_service_client = BlobServiceClient.from_connection_string(connection_string)
        container_client = blob_service_client.get_container_client(container_db)
        blob_client = container_client.get_blob_client(folder_db+"/"+file_to_file_to_db)
        start=time.time()
        #upload data
        block_list=[]
        chunk_size=4*1024*1024
        with  open(file_db_local_complete_path,'rb') as  f:
          while  True:
                read_data = f.read(chunk_size)
                if  not  read_data:
                    break  # done
                blk_id = str(uuid.uuid4())
                blob_client.stage_block(block_id=blk_id,data=read_data)
                block_list.append(BlobBlock(block_id=blk_id))

        blob_client.commit_block_list(block_list)
        end=time.time()
    except Exception as e:
        print(e)
        pass

The function uploads the file part by part, depending on the bandwidth and internet upload speed you can change it:

Full code (I will test it later because I have to set all the environment in Azure and local):

from smbprotocol.open import Open, CreateDisposition, FilePipePrinterAccessMask, FileAttributes, ShareAccess, ImpersonationLevel
from smbprotocol.session import Session
from smbprotocol.connection import Connection
import psutil
the_current_user=psutil.Process().username()

driveinnetwork = 'NetCOmpanyDrive'
username = the_current_user #this can be the_current_user, this may be a vulnerability, recommended to be used for IT Admin only
password = 'password' #if each user is managed with EntraID, we will need MSAL instead of the actual password, so we do not expose the real one, or the IT Admin can enable the script as a hidden service
shared_folder = 'shared_folder_drive'
file_path = '/path/to/localfile.pdf'

#or os.system(f"net use {driveinnetwork} /user:{username} {password}") #but this may be a security issue

#client side connector
connection = Connection(uuid.uuid4(), driveinnetwork, 445)
connection.connect()
session = Session(connection, username, password)
session.connect()

allids = session.tree_connect(shared_folder)

# Access a file
with Open(allids, file_path, desired_access=FilePipePrinterAccessMask.GENERIC_READ, create_disposition=CreateDisposition.FILE_OPEN, file_attributes=FileAttributes.FILE_ATTRIBUTE_NORMAL, share_access=ShareAccess.FILE_SHARE_READ, impersonation_level=ImpersonationLevel.Impersonation, oplock_level=0) as file:
    #read
    data=file.read() #can read all or by chunks for memory usage

#Add Azure POST file to Azure Function, more mechanisms may be needed here to know when remote tries to read local and vice-versa
    
def s_on_db(container_db,folder_db,file_db_local_complete_path,file_to_file_to_db):
    try:
        connection_string="BlobEndpoint=https://mystorageaccount2024.blob.core.windows.net/;QueueEndpoint=h..."
        blob_service_client = BlobServiceClient.from_connection_string(connection_string)
        container_client = blob_service_client.get_container_client(container_db)
        blob_client = container_client.get_blob_client(folder_db+"/"+file_to_file_to_db)
        start=time.time()
        #upload data
        block_list=[]
        chunk_size=4*1024*1024
        with  open(file_db_local_complete_path,'rb') as  f:
          while  True:
                read_data = f.read(chunk_size)
                if  not  read_data:
                    break  # done
                blk_id = str(uuid.uuid4())
                blob_client.stage_block(block_id=blk_id,data=read_data)
                block_list.append(BlobBlock(block_id=blk_id))

        blob_client.commit_block_list(block_list)
        end=time.time()
    except Exception as e:
        print(e)
        pass


#data will be sent to the cloud using the function s_on_db(container_db,folder_db,file_db_local_complete_path,file_to_file_to_db)

#close connection
session.tree_disconnect(tree_id)
session.logoff()
connection.disconnect()

The CPE option could be easier and keeps the Microsoft EntraID Authentication schema, requires a work or school account and some O365 Licenses (EntraID P1 or P2)

Answer 2

Seems to be an XY problem. Why do you want to list onprem files from an Azure function ?

I'm not sure this is even possible. But the real question is, do you really want to expose the whole locale corporate file share ? Even using the express route ?

Azure functions are a serverless Cloud natif deployment, that can connect safely and natively to Azure Blob storage.

My suggestions :

1. Use an Azure File share, that will be mounted to your Azure Function AND to your onprem services
2. Or deploy an API onprem with samba access that exposes a route that get file location. Your Azure function can then call the API to access this information.

Some related answer :

• Access file on Azure virtual windows machine from an Azure Function
• https://serverfault.com/questions/956854/access-smb-file-share-from-azure-function