Breaking it down part by part

1. Match every shared track between two playlists

MATCH path=(p:Playlist)-[r1:HAS_TRACK]->(track)<-[r2:HAS_TRACK]-(other:Playlist)

This is the same convergent pattern as before: playlist p and playlist other both point to the same track. Crucially, there is no position filter here, so it captures every track two playlists have in common, regardless of where each one sits. Each row produced by the MATCH represents one shared track together with its two relationships r1 and r2.

2. Aggregate the shared tracks into a list with WITH + collect

WITH 
    p AS playlistLeft, 
    other AS playlistRight,
    collect(
        {track: track, positionLeft: r1.position, positionRight: r2.position}
    ) AS commonTracks

The WITH clause groups the rows by the two playlists (p and other, renamed to playlistLeft and playlistRight for readability). Whatever is not an aggregate becomes the grouping key, so Cypher collapses all the rows for a given playlist pair into a single row. collect(...) then gathers every shared track into a list called commonTracks, where each element is a small map recording the track itself and its position in each playlist (positionLeft from r1, positionRight from r2).

3. Keep only meaningfully overlapping pairs

WHERE size(commonTracks) > 5

size(commonTracks) is the number of tracks the two playlists share. This filter discards pairs with five or fewer common tracks so the result focuses on genuinely similar playlists rather than incidental single-track overlaps.

4. Compute the similarity metrics with list comprehensions

RETURN 
    playlistLeft.name, 
    playlistRight.name,
    size([track in commonTracks 
      WHERE track.positionLeft = track.positionRight]) 
      AS tracksWithSamePosition,
    size([track in commonTracks 
      WHERE NOT track.positionLeft = track.positionRight]) 
      AS tracksAtDifferentPosition

Each [track in commonTracks WHERE ...] is a list comprehension that filters the collected list, and wrapping it in size(...) counts how many elements survive:

• tracksWithSamePosition counts shared tracks whose positionLeft equals positionRight: tracks in the same slot in both playlists.
• tracksAtDifferentPosition counts the rest: shared tracks that appear in different slots.

Together the two columns describe both how much two playlists overlap and how aligned that overlap is.

5. Order and cap the output

ORDER BY tracksWithSamePosition DESC
LIMIT 100;

Results are sorted so the most tightly aligned pairs (most tracks in identical positions) come first, and LIMIT 100 keeps the report to the top 100 pairs.

The result

The table returns one row per playlist pair, ranked by tracksWithSamePosition:

• The top two rows are “Folk Metal” and “Folk metal” (essentially the same playlist under two different capitalisations). They share 66 tracks in total (4 at the same position plus 62 at different positions), which strongly suggests they are duplicates or forks of one another.
• Because the pattern is symmetric, each pair appears twice with left and right swapped (rows 1 and 2, rows 3 and 4). The metrics are identical in both directions, confirming the counts are order-independent.
• The “Dangdut” / “dangdut” pair lower down shares far fewer tracks (2 aligned, 7 misaligned), so it ranks below the folk-metal pair even though it is also a near-duplicate name.

Notice that a high tracksAtDifferentPosition relative to tracksWithSamePosition (62 vs 4 for the folk-metal pair) tells us the two playlists contain almost the same music but in a reshuffled order, a subtle distinction that would be invisible if we only counted shared tracks. This kind of ranked, position-aware overlap is precisely what powers “playlists like this” recommendations directly from the graph.

Breaking it down part by part

1-3. Match, aggregate, and filter (unchanged)

The first three stages are identical to the ranking query in Ranking the Most Similar Playlists:

• the convergent MATCH finds every track two playlists share,
• the first WITH + collect groups those rows per playlist pair into a commonTracks list, and
• WHERE size(commonTracks) > 5 drops pairs with too little overlap.

See that section for the full explanation of these steps.

4. Reshape into similarity metrics with a second WITH

WITH 
    playlistLeft, 
    playlistRight,
    size([track in commonTracks 
      WHERE track.positionLeft = track.positionRight]) 
      AS tracksWithSamePosition,
    size([track in commonTracks 
      WHERE NOT track.positionLeft = track.positionRight]) 
      AS tracksAtDifferentPosition

This is the same pair of list comprehensions used in the earlier RETURN, but here they feed a second WITH instead of ending the query. The difference matters: WITH keeps the pipeline open, carrying playlistLeft, playlistRight, and the two counts forward so the next clause can act on them rather than just report them.

5. Create (or reuse) the SIMILAR relationship with MERGE

MERGE (playlistLeft)-[r:SIMILAR]->(playlistRight)

MERGE is the match-first-then-create clause: if a SIMILAR relationship already exists between this pair it is reused, otherwise a new one is created. Using MERGE here (rather than CREATE) makes the query idempotent, you can re-run it as the data changes without piling up duplicate similarity edges.

6. Store the scores as relationship properties with SET

SET 
    r.samePosition = tracksWithSamePosition, 
    r.notSamePosition = tracksAtDifferentPosition;

Finally SET writes the two counts onto the new relationship r as the properties samePosition and notSamePosition. The similarity is now a first-class part of the graph: a query like MATCH (p:Playlist)-[:SIMILAR]->(q) RETURN q retrieves related playlists in a single hop, with the alignment scores available for ranking, no re-aggregation required.

Reading the query

• MATCH path=(playlist1)-[:SIMILAR]-(playlist2) matches any two playlists connected by a SIMILAR relationship. Unlike the earlier queries, there is no HAS_TRACK traversal or aggregation here. The hard work of computing similarity was already done and persisted, so this is a single, cheap hop.
• The relationship is written without an arrow direction (-[:SIMILAR]- rather than ->). This makes the match undirected, so a pair is found regardless of which playlist was stored as the source or target.
• RETURN path returns the whole matched pattern so it renders as a graph, and LIMIT 100 caps the number of relationships drawn.

The result

Instead of a single pair, the Browser now shows a whole similarity network:

• Each blue node is a playlist and each SIMILAR edge means the two playlists share more than five tracks (the threshold set when the relationships were built).
• Some playlists act as hubs, for example Electro-popunk… sits at the centre of many incoming SIMILAR edges, meaning a lot of other playlists resemble it. These hubs are natural anchors for “if you like this, try these” recommendations.
• Smaller clusters also emerge, such as the Beach house → Kygo & Friends → Chill Deep House grouping on the left, which hangs together as a recognisable sub-genre neighbourhood.

Because similarity now lives in the graph as explicit edges, this entire view is produced by one short traversal rather than the multi-stage aggregation used to compute it. That is the payoff of persisting derived relationships: the expensive analysis runs once, and every later exploration is a fast walk over the stored network.

Reading the query

• MATCH (p:Playlist) WHERE p.name = "all that jazz" finds the one Playlist node whose name property equals all that jazz.
• RETURN p returns that node. Although the query only asks for the playlist itself, the Neo4j Browser expands a returned node to show its immediate relationships, which is why the surrounding tracks appear in the graph view.

The result

The blue node in the centre is the all that jazz playlist, surrounded by the green Track nodes it points to through HAS_TRACK relationships, songs such as Blowin’ The Blu…, When Sunny…, and Blue in Green. Two other blue Playlist nodes, smooth jazz and Funky / Jazz / B…, also appear because they are connected to all that jazz by the SIMILAR edges built in Persisting Similarity as a Relationship.

This view is essentially a snapshot of everything the recommendation query has to work with: the playlist’s own tracks (to exclude), and the similar playlists (to draw suggestions from). With that context in place, the next section walks through generating the actual recommendations.

Reading the query

• MATCH (p)-[r:HAS_TRACK]->(t) walks every HAS_TRACK relationship from a playlist p to a track t, exposing the position property on r ( the same property introduced in Finding Similarities in a Graph).
• COUNT {(p)-[:HAS_TRACK]->()} is a COUNT subquery: for each playlist it counts how many tracks that playlist has. Since positions run sequentially up to the total number of tracks, this count equals the position of the final track.
• WHERE r.position = COUNT {...} therefore keeps only the relationship whose position matches the track count, i.e. the very last track in each playlist.
• RETURN t returns those last tracks.

This is a more compact alternative to the head/tail approach used in Recommending the Next Track: instead of collecting all tracks in reverse order and taking the head, it computes the last position inline and filters straight to it.

The result

Each green node in the middle row, for example Shell Shocked and Hot Like Sauce, is the final track of a playlist, matched because its position equals that playlist’s total track count. The blue Playlist nodes above them are the playlists that end on those tracks, and the lower green nodes (Wiz Khalifa, Ty Dolla $ign, Pretty Lights, …) are the artists the Browser reveals when it expands each returned track through its ARTIST relationships.

Breaking it down part by part

1. Precompute the 10 most popular tracks with COLLECT { ... }

WITH COLLECT {
    MATCH (popularTrack:Track)-[:HAS_TRACK]-(:Playlist)
    WITH popularTrack, count(*) as playlistCount
    ORDER BY playlistCount DESC
    LIMIT 10
    RETURN popularTrack
} AS popularTracks

COLLECT { ... } is a subquery expression: it runs a full nested query and gathers its results into a list. Inside, the pattern counts how many playlists each track appears in (count(*)), orders by that count, and keeps the top 10. The outcome is a popularTracks list that later stages use as an exclusion set. Very popular tracks are poor recommendations precisely because everyone already has them.

2. Select the target playlist

MATCH (p:Playlist) WHERE p.name = "all that jazz"

A straightforward MATCH pins the query to a single playlist by name. This is the playlist we want recommendations for.

3. Collect the playlist’s tracks in reverse position order

WITH p, popularTracks,
COLLECT {
    MATCH (p)-[r:HAS_TRACK]->(t)
    WITH t, r
    ORDER BY r.position DESC
    RETURN t
} AS playlistTracks

Another COLLECT { ... } subquery gathers the playlist’s tracks, this time ordered by r.position descending so the most recently added track comes first. popularTracks is carried through untouched. This reuses the position property on HAS_TRACK introduced in Finding Similarities in a Graph.

4. Split the list into the last track and the rest with head / tail

WITH 
    p AS playlist, 
    popularTracks,
    head(playlistTracks) AS lastTrack,
    tail(playlistTracks) AS previousTracks

Because the list is in reverse order, head(playlistTracks) is the last track the listener added. The “current context” for the recommendation and tail(playlistTracks) is everything else, kept as previousTracks so those can be excluded later.

5. Hop to similar playlists that share the last track

MATCH (lastTrack)<-[:HAS_TRACK]-(otherPlaylist)-[:SIMILAR]-(playlist)
WHERE otherPlaylist <> playlist

This is the heart of the recommendation. It combines two hops: otherPlaylist must both contain the last track (lastTrack<-[:HAS_TRACK]-otherPlaylist) and be linked to our playlist by the SIMILAR edge built in Persisting Similarity as a Relationship. The WHERE otherPlaylist <> playlist guard stops the playlist from recommending itself. Reusing the stored SIMILAR relationship is what keeps this cheap, no similarity is recomputed here.

6. Gather candidate tracks, minus what the listener already has or knows

MATCH (otherPlaylist)-[:HAS_TRACK]->(recommendation)
WHERE NOT recommendation IN previousTracks
AND NOT recommendation IN popularTracks

From each similar playlist, take its tracks as recommendation candidates, then filter out anything already in previousTracks (the listener has it) or in popularTracks (too generic to be a useful suggestion).

7. Score by frequency and return the top 10

RETURN recommendation.id AS recommendedTrackId, 
recommendation.name AS recommendedTrack, 
otherPlaylist.name AS fromPlaylist,
count(*) AS score
ORDER BY score DESC
LIMIT 10

count(*) aggregates how often each candidate appears across the matched similar playlists: the more independent playlists suggest a track, the higher its score. Sorting by score descending and taking the top 10 yields the final recommendation list, along with the playlist each suggestion came from.

The result

The query returns a ranked table of suggestions for the all that jazz playlist:

• Each row is a recommendedTrack, for example Stompin’ at the Savoy, I May Be Wrong, and My One And Only Love, none of which are already in the playlist or in the global top-10 popular tracks.
• The fromPlaylist column shows they all surfaced via smooth jazz, a playlist connected to all that jazz by a SIMILAR edge and sharing its last track. This confirms the traversal followed the intended path.
• The score of 2 means each track appeared twice across the matched similar playlists. With a small, tightly related neighbourhood the scores are low and close together; on a larger graph this frequency count is what separates strong recommendations from incidental ones.

Taken together, this query shows the whole point of the earlier steps paying off: the precomputed SIMILAR relationships turn what would be an expensive multi-join recommendation into a short, readable traversal that produces genuine “you might also like” suggestions.

Reading the query

• MATCH (t1:Track {id: "..."}) and MATCH (t2:Track {id:"..."}) pin down the two endpoint tracks by their unique id property. Matching on id (rather than name) guarantees exactly one node each.
• MATCH path = SHORTEST 5 (t1)-[r:HAS_TRACK]-+(t2) is the key part:
  • SHORTEST 5 asks for the five shortest paths between the two tracks, ordered by length.
  • -[r:HAS_TRACK]-+ is a variable-length, undirected pattern. The + means “one or more HAS_TRACK hops”, and the lack of an arrow lets the path traverse the relationship in either direction: track → playlist → track → playlist …, so it can weave through the shared playlists that connect the two songs.
• RETURN path returns the matched paths so the Browser draws them.

Because tracks only connect through playlists, every step alternates between a Track and a Playlist, reusing the same HAS_TRACK relationship seen throughout this post, which is read here in both directions rather than just playlist → track.

The result

The graph shows the two endpoint tracks connected via the playlists that contain them. The blue Playlist nodes smooth jazz and all that jazz sit at the top, each fanning out through HAS_TRACK edges to the green Track nodes below (Now See How Y…, Journey Into M…, Georgia On My…, Greensleeves, Skating In Cent…, When Your Lo…). A path hops track → playlist → track, so the two playlists act as the bridges that join the endpoints in just a few steps. Returning several shortest paths at once reveals how many independent short connections exist, not just that a single one does, which is a compact way to gauge how closely two tracks are related within the graph.

Reading the query

• The two endpoint MATCH clauses are identical to Connecting Two Tracks with a Shortest Path: they pin t1 and t2 by id.
• MATCH p = ((t1)-[*..5]-(t2)) is the difference. -[*..5]- is an undirected variable-length pattern meaning “between one and five relationships of any type”, and most importantly there is no SHORTEST keyword. Without it, Cypher returns every path up to length 5, not just the shortest ones.
• RETURN p returns all of those paths.

How it differs from the shortest-path block

In short, the first block asks a targeted question (“give me the few closest connections through shared playlists”), while the second asks an exhaustive one (“show me everything reachable within five hops”). The exhaustive form is useful for exploring a neighbourhood, but it can return a large, dense result because the number of paths grows quickly with each extra hop.

The result

The difference is immediately visible. Rather than a handful of clean bridges, the query streams back 330 records and fills the view with a dense web of nodes. The two blue Playlist hubs: Funky / Jazz / B… and all that jazz, each fan out through dozens of HAS_TRACK edges to green Track nodes such as Lonely Woman, Blues Walk, Naima, and The Final Come… (one of the highlighted endpoints). Because the pattern accepts any relationship of up to five hops, it captures the entire cluster of tracks reachable through those shared playlists, not just the shortest routes between the two endpoints. This illustrates why SHORTEST matters: the same neighbourhood, filtered to the few shortest paths, is far easier to interpret than the full enumeration shown here.

A single node is a pattern too

Consider merging one artist node:

MERGE (n:Artist {name:"Wham!", origin:"UK"})

For this MERGE to match an existing node (and therefore not create another one), your graph must already contain:

• a node labeled Artist (other labels may exist and do not matter), and
• a property name with value "Wham!" and a property origin with value "UK" (other properties may exist and do not matter).

If the artist node in your graph has the property name with value "Wham!" but no origin property, the pattern does not match, so a new Artist node with both properties is created. You now have two Wham! artist nodes (assuming a uniqueness constraint did not fail first). The lesson: every property you put inside a MERGE becomes part of the key it must match on.

Patterns with relationships match as a whole

The same all-or-nothing rule applies to patterns that contain relationships. Suppose the artist Wham! already exists and you want to link a track to it:

MERGE (t:Track {name:"Last Christmas"})-[:ARTIST]->(a:Artist {name:"Wham!"})

It is tempting to assume this will reuse the existing Wham! node and only create the track and the ARTIST relationship. It will not. Because the track "Last Christmas" is not yet in the graph, the entire pattern fails to match, so MERGE creates the entire pattern, including a second Wham! node.

Keep it simple: merge nodes first, then relationships

The reliable approach is to MERGE each node individually on its key, leaving off the other properties, and then use those matched nodes to MERGE the relationship between them:

MERGE (t:Track {name:"Last Christmas"})
MERGE (a:Artist {name:"Wham!"})
MERGE (t)-[:ARTIST]->(a)

Here each node is matched (or created) on its own key first, so the existing Wham! node is reused, and only then is the ARTIST relationship merged between the two known nodes. Breaking a compound pattern into separate single-node merges followed by a relationship merge is the safest way to keep your graph free of duplicates.

Reading the query line by line

• MATCH path=(user)-[:OWNS]->(p:Playlist) starts the pattern at a user node that OWNS a Playlist, binding the playlist to p.
• -[:HAS_TRACK]->(track:Track) follows the playlist’s HAS_TRACK relationship to the Track it contains, binding it to track.
• --(albumOrArtist) traverses one more relationship from that track in any direction and of any type (the bare -- has no arrow and no [:TYPE]), matching either the Album linked by HAS_TRACK or the Artist linked by ARTIST. The neighbouring node is bound to albumOrArtist.
• path=(...) captures the whole matched chain, and RETURN path renders it as a connected subgraph in the Neo4j Browser.

Why the merged track matters

In Figure 1 the central Track node (“Song 2 - 2012 Remastered Version”) is reached from both sides at once: the bleapkin user’s playlist Pixies — Wher... points into it via HAS_TRACK, while the album Blur [Special Edition] points into it via HAS_TRACK and the track points out to the artist Blur via ARTIST. The node details panel shows its id, 1FTSo4v6BOZH9QxKc3MbVM, exactly the track_id shared by both CSV files.

This is the payoff of using MERGE: earlier, with CREATE, that same track_id produced two separate Track nodes and left the playlist and album/artist subgraphs disconnected. Because both imports now MERGE on {id: row.track_id}, the second import matched the track created by the first instead of duplicating it, so a single shared node joins the two subgraphs into one connected graph.

Using uniqueness constraints instead

We replace each IS NODE KEY with IS UNIQUE, which Community Edition supports for every label:

//008-index-creation.cypher
// Community Edition: uniqueness constraints (unique only, not existence)
CREATE CONSTRAINT playlist_id IF NOT EXISTS
FOR (n:Playlist) REQUIRE n.id IS UNIQUE;
CREATE CONSTRAINT user_id IF NOT EXISTS
FOR (n:User) REQUIRE n.id IS UNIQUE;
CREATE CONSTRAINT track_id IF NOT EXISTS
FOR (n:Track) REQUIRE n.id IS UNIQUE;
CREATE CONSTRAINT album_id IF NOT EXISTS
FOR (n:Album) REQUIRE n.id IS UNIQUE;
CREATE CONSTRAINT artist_id IF NOT EXISTS
FOR (n:Artist) REQUIRE n.id IS UNIQUE;

A few things worth noting:

• A uniqueness constraint automatically creates a backing range index on id, so our MERGE (n {id: ...}) lookups stay fast without a separate CREATE INDEX.

• IF NOT EXISTS makes the script safe to run more than once.

• Community Edition cannot enforce the existence half, but our imports already set id on every MERGE, so the key is always present. To double-check, we can look for any node missing the key:

  MATCH (n:Track) WHERE n.id IS NULL RETURN count(n);

Why constraints speed up importing a larger CSV

With the uniqueness constraints in place, we can import a much larger tracks file, sample_tracks_medium.csv, using exactly the same MERGE logic as before:

//import-larger-csv-file-tracks-medium.cypher
LOAD CSV WITH HEADERS FROM "file:///medium/sample_tracks_medium.csv" AS row

MERGE (track:Track {id: row.track_id})
SET track.uri = row.track_uri, track.name = row.track_name

MERGE (album:Album {id: row.album_id})
SET album.uri = row.album_uri, album.name = row.album_name

MERGE (artist:Artist {id: row.artist_id})
SET artist.uri = row.artist_uri, artist.name = row.artist_name

MERGE (album)-[:HAS_TRACK]->(track)
MERGE (track)-[:ARTIST]->(artist);

Every MERGE (track:Track {id: row.track_id}) has to first match whether a Track with that id already exists before deciding to create it. Without an index, Neo4j has no shortcut for that lookup: it must scan every existing Track node and compare its id, which is an operation. As the graph grows, each successive row scans a larger set of nodes, so the total cost of importing rows grows roughly as . For the tiny sample file this is unnoticeable, but for sample_tracks_medium.csv, with many more rows, that quadratic behaviour quickly makes the import slow.

Creating the uniqueness constraint first fixes this. As noted above, a uniqueness constraint automatically builds a backing range index on the id property. With that index, each MERGE lookup becomes an indexed seek (roughly instead of a full scan) so importing rows drops from about to about . The larger the CSV, the bigger this saving, which is why creating the constraints (and their indexes) before the bulk import is the key to fast ingestion. The constraint also guarantees that no duplicate Track, Album, or Artist slips in while the larger file is loading.

We then import the matching larger playlists file. This time we fix the position data type during ingestion rather than in a separate pass:

//merge-large-playlists-and-cast-data-type-during-ingestion.cypher
LOAD CSV WITH HEADERS FROM "file:///medium/sample_playlists_medium.csv" AS row

MERGE (playlist:Playlist {id: row.id})
SET playlist.name = row.name

MERGE (user:User {id: row.user_id})
MERGE (track:Track {id: row.track_id})

MERGE (user)-[:OWNS]->(playlist)
MERGE (playlist)-[:HAS_TRACK {position: toInteger(row.track_index)}]->(track)

Recall from the smaller import that LOAD CSV reads every column as text, so row.track_index arrives as a string like "0" or "1". Earlier we had to run a second query (SET r.position = toInteger(r.position)) to repair the type after the fact. Here we wrap the value in toInteger(row.track_index) inside the MERGE, so the conversion happens as each relationship is created and the position is stored as an integer from the outset.

Doing the cast inside the MERGE also matters for correctness, not just convenience. MERGE matches on the exact pattern, including relationship properties, so MERGE (playlist)-[:HAS_TRACK {position: toInteger(row.track_index)}]->(track) looks for a HAS_TRACK relationship whose position is the integer value. Had we merged on the raw string row.track_index, re-running the import (or running it alongside the earlier integer-typed data) could fail to match the existing relationship and create a duplicate with a differently typed position. Converting the value before it is used as part of the merge key keeps the type consistent and the relationship de-duplicated.

After importing the larger tracks and playlists files, viewing the whole database with MATCH (n) OPTIONAL MATCH (n)-[r]->(o) RETURN * shows a much denser graph than the small sample, with many more Track, Album, Artist, Playlist, and User nodes and their relationships:

Recommended approach: use the Neo4j import folder

The simplest and safest fix is to copy the CSV into Neo4j’s import directory and reference it by name only:

sudo cp /path/to/your/sample_tracks.csv /var/lib/neo4j/import/
sudo chown neo4j:neo4j /var/lib/neo4j/import/sample_tracks.csv

Then load it in Cypher using a bare file:/// URL:

LOAD CSV WITH HEADERS FROM 'file:///sample_tracks.csv' AS row
RETURN row
LIMIT 5;

Cypher fundamentals

Cypher is Neo4j’s query language, and it describes graph patterns using an ASCII-art syntax where nodes are drawn as parentheses () and relationships as arrows -->. A few core concepts explain what the query above does:

• Nodes are the entities in the graph. (track) introduces a node and binds it to the variable track, so we can refer to it later in the same query.
• Labels classify nodes by type. In (track:Track), Track is the label, the equivalent of a table name in a relational database. Labels let us group and index nodes, for example “all Track nodes”.
• Properties are key/value pairs stored on a node (or relationship), written inside curly braces {}. They are analogous to columns on a row.
• Variables such as track and row are temporary names scoped to the query. row comes from LOAD CSV and represents the current CSV line, so row.track_id reads the track_id column of that line.

Reading the query line by line

CREATE (track:Track {id: row.track_id})
SET track.uri = row.track_uri,
    track.name = row.track_name

• CREATE (track:Track {id: row.track_id}) creates a new node, labels it Track, and sets its id property to the track_id value from the current CSV row. Setting the identifying property inline at creation time is a common idiom.
• SET track.uri = row.track_uri, track.name = row.track_name assigns two more properties to that same node: uri (from track_uri) and name (from track_name). SET can add or overwrite any number of properties.

CREATE vs. MERGE

CREATE always inserts a brand-new node, so if the query runs more than once, or the CSV contains duplicate track_id values (our sample data does), you will end up with duplicate Track nodes. When an id should be unique, use MERGE instead, which matches an existing node or creates it if none is found:

LOAD CSV WITH HEADERS FROM 'file:///sample_tracks.csv' AS row
MERGE (track:Track {id: row.track_id})
SET track.uri = row.track_uri,
    track.name = row.track_name

CREATE CONSTRAINT track_id_unique IF NOT EXISTS
FOR (t:Track) REQUIRE t.id IS UNIQUE;

How relationships are written

In Cypher, a relationship is drawn as an arrow between two nodes, with the relationship type in square brackets:

(startNode)-[:TYPE]->(endNode)

• The parentheses () refer to nodes. Here album, track, and artist are variables bound to nodes created (or matched) earlier in the same query.
• The square brackets [:HAS_TRACK] and [:ARTIST] hold the relationship type, the label that describes what the connection means. By convention relationship types are written in UPPER_SNAKE_CASE.
• The arrow -> gives the relationship a direction, pointing from the start node to the end node.

Reading the query line by line

• CREATE (album)-[:HAS_TRACK]->(track) creates a HAS_TRACK relationship directed from the album node to the track node, expressing that the album contains that track.
• CREATE (track)-[:ARTIST]->(artist) creates an ARTIST relationship directed from the track node to the artist node, expressing who performed the track.

Together these two statements wire each track into the graph: every track is linked upward to the album it belongs to and across to the artist who made it. The result is a connected structure such as (album)-[:HAS_TRACK]->(track)-[:ARTIST]->(artist) that we can later traverse, for example to find all tracks by an artist or every artist featured on an album.

MERGE (album)-[:HAS_TRACK]->(track)
MERGE (track)-[:ARTIST]->(artist);

Querying the Graph

With the data imported, we can query the graph to explore the connections we built. The following query returns the full album, track, and artist paths:

MATCH path=(artist:Artist)<-[:ARTIST]-(t:Track)<-[:HAS_TRACK]-(album:Album)
RETURN path

Reading the query line by line

• MATCH (artist:Artist)<-[:ARTIST]-(t:Track)<-[:HAS_TRACK]-(album:Album) describes a pattern to search for: an Album that HAS_TRACK a Track, which in turn points via ARTIST to an Artist. Neo4j finds every part of the graph that matches this shape.
• The arrows show relationship direction. We created the edges as (album)-[:HAS_TRACK]->(track) and (track)-[:ARTIST]->(artist), so reading from the artist the arrows point backwards (<-). The pattern above is simply those same relationships traversed in the opposite direction; it matches exactly the edges we inserted.
• path=(...) assigns the whole matched pattern to the variable path. A path is the ordered sequence of nodes and relationships that satisfied the match.
• RETURN path outputs each matched path. In the Neo4j Browser this renders as a visual subgraph of the connected Album, Track, and Artist nodes.

Because the pattern names all three labels and both relationship types, the query returns only fully connected album -> track -> artist chains, which is a quick way to confirm the import wired everything together correctly.

Inspecting node details

Clicking a node in the Neo4j Browser opens the Node details panel on the right, which lists the selected node’s label and every property stored on it. In Figure 2 the selected Album node shows:

• <id>: the internal element id Neo4j assigns to every node (for example 4:40548a74-6728-4cca-a57c-f9c184371f3e:1). It uniquely identifies the node inside the database and is generated automatically, distinct from the business id we imported.
• id: the property we set from the CSV ("7HvIrSkKGJCzd8AKyjTJ6Q", the Spotify album id). This is the value the MERGE/CREATE pattern keys on.
• name: the human-readable album title, "Blur [Special Edition]".
• uri: the Spotify URI, "spotify:album:7HvIrSkKGJCzd8AKyjTJ6Q".

These are exactly the three properties (id, name, uri) we assigned when creating Album nodes, so the panel is a convenient way to verify that the import populated each node correctly. Selecting the Track or Artist node instead would show its own label and the corresponding properties.

The problem with duplicated nodes

Looking at Figure 3, the graph doesn’t look quite right: it shows two separate chains that should be joined. Each chain comes from a different CSV file:

• The first pattern represents a row from the playlists file: (:User)-[:OWNS]->(:Playlist)-[:HAS_TRACK]->(:Track)
• The second pattern represents a row from the tracks file: (:Album)-[:HAS_TRACK]->(:Track)-[:ARTIST]->(:Artist)

The Track in both patterns is the same track and should therefore be the same node. Because we used CREATE, however, Neo4j blindly created a brand-new Track node for each file, leaving two disconnected copies instead of one shared node linking the two subgraphs.

We can confirm this by clicking the Track node in each chain and comparing the Node details panel. Their internal <id> values differ (they are two distinct nodes), but their id property is identical:

Both nodes carry id: "1FTSo4v6BOZH9QxKc3MbVM", which is exactly the value of the track_id column for that song in both CSV files, the playlists file and the tracks file. Since we set {id: row.track_id} when creating the node in each import script, the same track_id produced two separate nodes sharing one business id. That shared track_id is precisely the key we can use to recognise the row as the same track and collapse the duplicates into a single node.

Merging safely to avoid duplicates

CREATE simply creates a node or relationship as instructed. But every track in the CSV has a unique track id, and if a track already exists in the graph we do not want to create it again. The fix is to use MERGE instead of CREATE.

MERGE is a combination of MATCH and CREATE: it tries to find the pattern in its entirety, and if it already exists, nothing is created. Only if the pattern cannot be matched is the whole pattern created. To avoid duplicate Track nodes, MERGE needs a reliable key to match on first, and the track’s id (like the album and artist ids) is unique, so it can be used to determine whether the node already exists:

MERGE (track:Track {id: row.track_id})
SET track.uri = row.track_uri,
    track.name = row.track_name

Written this way, loading the playlists file will match the existing Track node created by the tracks file (rather than making a second one), so both patterns attach to a single shared node and the graph becomes properly connected.

CREATE CONSTRAINT track_id_unique IF NOT EXISTS
FOR (t:Track) REQUIRE t.id IS UNIQUE;

Reading the query line by line

• MATCH (n) finds every node in the database. MATCH is Cypher’s read clause for locating existing data, and the pattern (n) is a node with no label and no properties, so it matches all nodes and binds each one to the variable n.
• DETACH DELETE n deletes those nodes. The DETACH keyword first removes any relationships attached to each node, then deletes the node itself.

The DETACH part is essential: Neo4j refuses to delete a node that still has relationships, because that would leave dangling edges. A plain DELETE n on a connected node fails with an error, whereas DETACH DELETE removes the node and its relationships together. Since MATCH (n) selects everything, this query empties the graph completely, giving you a clean slate to re-run the import.

Physical parameters

Both pendulum arms have equal length ( m) and equal mass ( kg). The time window is s with 1,000 evaluation points. The initial state vector is , so the upper arm starts at and the lower arm hangs vertically, both from rest.

Original second-order equations

Converting to a first-order system

odeint requires first-order ODEs, but the double pendulum equations are second-order. We introduce the angular velocities and as new variables, turning two second-order equations into four first-order equations.

Step 1: Define two new variables:

This immediately gives two of the four first-order equations:

Step 2: Since and , the two original second-order equations become equations for and . Substituting () and ( into the boxed equations:

Step 3: These two equations are linear in the unknowns and (everything else such as , , , is known at the current time step). Writing them as a matrix system:

Solving this system (e.g. by substitution or Cramer’s rule) gives explicit expressions for and , which is what the code computes via den1, den2 and the numerator expressions.

Result: The full first-order system passed to odeint is:

The state vector is and pendulum_system returns .

Solving for and

The two boxed equations each contain both () and (), so they must be solved simultaneously as a linear system in the accelerations. The code does this algebraically:

• delta : the relative angle between the two arms.
• den1 : the determinant-like denominator that arises from eliminating to solve for . It can never be zero for physical parameters since and .
• den2 den1: the corresponding denominator for , obtained by eliminating .
• The numerators collect the gravitational (), centripetal (), and coupling () terms after the elimination.

Output

The returned solution array has shape (1000, 4), where columns 0-3 hold , , , at each time point.

g       = 9.81
L1, L2  = 1.0, 1.0
m1, m2  = 1.0, 1.0

t_start, t_end = 0, 5
t = np.linspace(t_start, t_end, 1000)

# Initial conditions: [theta1, theta2, omega1, omega2]
initial_state = [45 * np.pi / 180, 0.0, 0.0, 0.0]

# Define the system of ODEs for the double pendulum
def pendulum_system(states, t, g, L1, L2, m1, m2):
    t1, t2, w1, w2 = states
    dt1 = w1
    dt2 = w2
    delta = t2 - t1
    den1  = (m1 + m2) * L1 - m2 * L1 * np.cos(delta)**2
    dw1   = (m2 * L1 * w1**2 * np.sin(delta) * np.cos(delta) +
             m2 * g * np.sin(t2) * np.cos(delta) +
             m2 * L2 * w2**2 * np.sin(delta) -
             (m1 + m2) * g * np.sin(t1)) / den1
    den2  = (L2 / L1) * den1
    dw2   = (-m2 * L2 * w2**2 * np.sin(delta) * np.cos(delta) +
             (m1 + m2) * g * np.sin(t1) * np.cos(delta) -
             (m1 + m2) * L1 * w1**2 * np.sin(delta) -
             (m1 + m2) * g * np.sin(t2)) / den2
    return [dt1, dt2, dw1, dw2]

solution = odeint(
    pendulum_system,
    initial_state,
    t,
    args=(g, L1, L2, m1, m2))
theta1 = solution[:, 0]
theta2 = solution[:, 1]
omega1 = solution[:, 2]
omega2 = solution[:, 3]

print(f"Numerical solution computed over t ∈ [{t_start}, {t_end}] s")
print(f"theta1 range: [{theta1.min():.3f}, {theta1.max():.3f}] rad")
print(f"theta2 range: [{theta2.min():.3f}, {theta2.max():.3f}] rad")

Numerical solution computed over t ∈ [0, 5] s
theta1 range: [-0.427, 0.785] rad
theta2 range: [-1.144, 0.622] rad

Displacement and Phase Portrait Plots

• Displacement plot (, vs ): shows both angular displacements over 5 seconds. Unlike the single damped pendulum (which decays monotonically), there is no damping here, so neither amplitude decays. The two curves oscillate with different frequencies and amplitudes because energy transfers back and forth between the arms through the coupling terms. starts at and starts at , but the lower arm quickly picks up motion from the upper arm. The resulting pattern is quasi-periodic – it looks irregular but repeats (approximately) over a longer timescale.

• Phase portrait ( vs for each arm): plots angular velocity against displacement with time removed. For a single undamped pendulum this would be a closed ellipse; here, the coupling between the two arms distorts the trajectories. Each arm traces a complex, non-repeating loop because its motion is modulated by the other arm. The trajectories remain bounded (no runaway growth), confirming that the system is in the regular (non-chaotic) regime at these initial conditions.

# ---------------------------------------------------------------------------
#  Plot 1: Displacement plot 
plt.figure(figsize=(9, 4))

# Plot the first arm's angle with a solid line
plt.plot(
    t,
    theta1,
    label=r'$\theta_1$ — Top Arm')

# Plot the second arm's angle with a dashed line
plt.plot(
    t,
    theta2,
    label=r'$\theta_2$ — Bottom Arm',
    linestyle='--')
plt.xlabel('Time (s)', fontsize=12)
plt.ylabel('Angular Displacement (rad)', fontsize=12)
plt.title('Double Pendulum — Numerical Solution', fontsize=13)
plt.legend(fontsize=11)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

# ---------------------------------------------------------------------------
#  Plot 2: Phase portrait 
plt.figure(figsize=(9, 4))
plt.plot(
    theta1,
    omega1,
    label=r'Arm 1 ($\theta_1$ vs ${d\theta_1}/{dt}$)')
plt.plot(
    theta2,
    omega2,
    label=r'Arm 2 ($\theta_2$ vs ${d\theta_2}/{dt}$)',
    alpha=0.6)
plt.xlabel(
    'Angular Displacement (rad)', fontsize=12)
plt.ylabel('Angular Velocity (rad/s)', fontsize=12)
plt.title('2-DOF Phase Portrait', fontsize=13)
plt.legend(fontsize=11)
plt.grid(True, alpha=0.3)
plt.tight_layout()
plt.show()

Sampling strategy

250 time points are drawn uniformly at random from s and sorted into chronological order. The ODE is solved at exactly these times to obtain the “true” displacement values for both arms. Using random (non-uniform) spacing mimics real sensor data, where measurements may arrive at irregular intervals.

Adding noise

Each displacement value is perturbed independently with Gaussian noise of rad (). Crucially, and receive separate noise draws – the measurement errors on the two sensors are independent, which is physically realistic (two separate angle sensors would have uncorrelated noise).

Overlay plots

Two subplots compare the 1,000-point numerical solution (coloured curve) with the 250 noisy data points (black scatter) for each arm separately:

• Top subplot (): the noisy data points should scatter tightly around the blue curve. Since has moderate amplitude ( rad peak), rad is small relative to the signal – the noise is barely visible.

• Bottom subplot (): the noisy data points scatter around the red curve. has larger amplitude than , so the same absolute noise level is an even smaller fraction of the signal here.

Plotting the two arms separately (rather than on one axis) makes it easier to verify that the noise level is realistic for each angle independently, since their amplitude ranges differ.

# For reproducibility
np.random.seed(42)

 # number of experimental observation points
N_exp_points = 250  

# noise standard deviation (rad)
exp_noise    = 0.005

# Generate synthetic experimental data by sampling the numerical 
# solution at random time points and adding noise
# sort to ensure time points are in order for plotting
t_exp_np = np.sort(np.random.rand(N_exp_points) * (t_end - t_start) + t_start)

# Integrate the system at the experimental time points to get the 
# "true" angles, then add noise
solution_exp  = odeint(
    pendulum_system,
    initial_state,
    t_exp_np,
    args=(g, L1, L2, m1, m2))
theta1_exp_np = solution_exp[:, 0] + exp_noise * np.random.randn(N_exp_points)
theta2_exp_np = solution_exp[:, 1] + exp_noise * np.random.randn(N_exp_points)

# ---------------------------------------------------------------------------
fig, axes = plt.subplots(2, 1, figsize=(9, 8))
axes[0].plot(
    t,
    theta1,
    color='blue',
    label=r'$\theta_1$ Numerical',
    alpha=0.7)
axes[0].scatter(
    t_exp_np,
    theta1_exp_np,
    color='black',
    s=10,
    label=r'$\theta_1$ Experimental')
axes[0].set_xlabel('Time (s)')
axes[0].set_ylabel('Angle (rad)')
axes[0].set_title('Top Pendulum')
axes[0].legend(fontsize=10)
axes[0].grid(True, alpha=0.3)

axes[1].plot(
    t,
    theta2,
    color='red',
    label=r'$\theta_2$ Numerical',
    alpha=0.7)
axes[1].scatter(
    t_exp_np,
    theta2_exp_np,
    color='black',
    s=10,
    label=r'$\theta_2$ Experimental')
axes[1].set_xlabel('Time (s)')
axes[1].set_ylabel('Angle (rad)')
axes[1].set_title('Bottom Pendulum')
axes[1].legend(fontsize=10)
axes[1].grid(True, alpha=0.3)

plt.suptitle(
    r'Synthetic Experimental Data vs Numerical Solution',
    fontsize=16)
plt.tight_layout()
plt.show()

Time normalisation

The raw experimental times are mapped to :

Displacement normalisation (per-angle)

Each angle is standardised independently using its own experimental statistics:

Separate statistics are essential because and have different amplitude ranges and means. Using the same constants for both would bias one angle’s normalised values away from zero mean and unit variance, distorting the data loss balance between the two outputs.

Tensor conversion

• t_exp_norm has shape (250, 1): a column vector of normalised times, matching the network’s single input.
• theta_exp_norm has shape (250, 2): the two normalised angles are stacked column-wise using torch.stack(..., dim=1), matching the network’s two outputs. Column 0 holds and column 1 holds .

# ----------------------------------------------------------------------------
# Time normalisation constants
t_min = t_start
t_max = t_end
dt    = t_max - t_min
t_exp_norm_np = 2 * (t_exp_np - t_min) / dt - 1.0

t_exp_norm    = torch.from_numpy(
    t_exp_norm_np).float().view(-1, 1).to(device)
print(f"t_exp_norm shape: {t_exp_norm.shape}")
print(f"t_exp_norm range (should be near [-1, +1]): " 
      + f"[{t_exp_norm.min():.3f}, {t_exp_norm.max():.3f}]")
print()
# ----------------------------------------------------------------------------
# Displacement normalisation: separate stats for each angle
theta1_mean = np.mean(theta1_exp_np)
theta1_std  = np.std(theta1_exp_np)
theta2_mean = np.mean(theta2_exp_np)
theta2_std  = np.std(theta2_exp_np)

# Standardise theta1 using its own statistics
theta1_exp_norm_np = (theta1_exp_np - theta1_mean) / theta1_std

# Standardise theta2 using its own statistics
theta2_exp_norm_np = (theta2_exp_np - theta2_mean) / theta2_std

# Convert to tensors, stack into shape (N, 2)
theta_exp_norm = torch.stack([
    torch.from_numpy(theta1_exp_norm_np).float(),
    torch.from_numpy(theta2_exp_norm_np).float()
], dim=1).to(device)

print(f"theta1_exp_norm mean (should be close to 0.0):  " 
      + f"{theta_exp_norm[:, 0].mean():.4f}")
print(f"theta2_exp_norm mean (should be close to 0.0):  " 
      + f"{theta_exp_norm[:, 1].mean():.4f}")
print(f"theta_exp_norm shape: " 
      f"{theta_exp_norm.shape}")

t_exp_norm shape: torch.Size([250, 1])
t_exp_norm range (should be near [-1, +1]): [-0.990, 0.974]

theta1_exp_norm mean (should be close to 0.0):  0.0000
theta2_exp_norm mean (should be close to 0.0):  0.0000
theta_exp_norm shape: torch.Size([250, 2])

Neural network architecture

FullyConnectedNN is a fully connected feedforward network with Tanh activations and two outputs, one for each pendulum angle. It takes a single normalised time input and returns normalised predictions for both angles simultaneously:

Component	Code	Description
fcs	Linear(1, 64) + Tanh	Input layer: maps the 1-D normalised time to 64 hidden units.
fch	3 × Linear(64, 64) + Tanh	Hidden layers: three additional layers (N_LAYERS - 1 = 3) for greater representational capacity.
fce	Linear(64, 2)	Output layer: maps to two normalised displacement values (no activation).

Normalisation helpers

• normalise_t(t_tensor): maps raw physical time to via

  

• denormalise_theta(theta_tensor): reverses the standardisation per-channel:

  • column 0 is denormalised with and
  • column 1 with .
  • This ensures the physics residual is evaluated in correct physical units for each angle independently.

Curriculum training strategy

PINNs often struggle to learn oscillatory solutions over long time horizons because errors in the physics residual accumulate from early to late times. Curriculum training mitigates this by training on progressively longer time windows:

Stage	Time window	Epochs	Purpose
1	s	5,000	Learn the initial transient accurately.
2	s	5,000	Extend the learned dynamics one second further.
3	s	5,000	Continue building outward from the established solution.
4	s	5,000	Near-full domain coverage.
5	s	5,000	Final stage covers the entire time domain.

At each stage, the collocation points and experimental data are restricted to the current window, but the network retains its weights from the previous stage. This means each stage refines and extends, rather than relearns the solution. The total training budget is 25,000 epochs across all five stages.

Training configuration

• Architecture: 1 input → 4 hidden layers of 64 Tanh units → 2 outputs.
• Collocation points: 1,000 uniformly spaced points within each stage’s time window, with requires_grad=True for autograd-based derivative computation. These are regenerated at the start of each curriculum stage.
• IC point: a single point at for the initial condition loss (four conditions: , , , ).
• Loss weights: , , . The reduced physics weight prevents the large-magnitude ODE residual (which involves terms scaled by ) from dominating the loss early in training, allowing the data loss to guide the network toward the correct waveform shape first.
• Optimiser: Adam with LR .
• LR scheduler: StepLR halves the learning rate every 5,000 epochs (aligning with curriculum stage boundaries), giving progressively finer convergence in later stages.

torch.manual_seed(42)

# -------------------------------------------------------------------
# PINN network architecture: fully connected with Tanh activations

class FullyConnectedNN(nn.Module):
    def __init__(self, N_INPUT, N_OUTPUT, N_HIDDEN, N_LAYERS):
        super().__init__()

        activation = nn.Tanh

        # Input layer + first hidden layer
        self.fcs = nn.Sequential(
            nn.Linear(N_INPUT, N_HIDDEN), activation())

        # Hidden layers (N_LAYERS - 1 because the first hidden
        # layer is already defined above)
        self.fch = nn.Sequential(*[
            nn.Sequential(
                nn.Linear(N_HIDDEN, N_HIDDEN), activation())
            for _ in range(N_LAYERS - 1)])

        # Output layer (linear, no activation)
        self.fce = nn.Linear(N_HIDDEN, N_OUTPUT)

    def forward(self, x):
        x = self.fcs(x)
        x = self.fch(x)
        x = self.fce(x)
        return x

# -------------------------------------------------------------------
# Normalisation helpers
def normalise_t(t_tensor):
    return 2.0 * (t_tensor - t_min) / dt - 1.0

def denormalise_theta(theta_tensor):
    # Reverses normalisation independently for each output channel
    th1 = theta_tensor[:, 0:1] * theta1_std + theta1_mean
    th2 = theta_tensor[:, 1:2] * theta2_std + theta2_mean
    return torch.cat([th1, th2], dim=1)

# -------------------------------------------------------------------
# Hyperparameters
n_input  = 1
n_output = 2       # predicts [theta1, theta2] simultaneously
n_hidden = 64
n_layers = 4
learning_rate = 1e-3

lambda_ic = 1
lambda_physics  = 0.1
lambda_data     = 1

# -------------------------------------------------------------------
# Curriculum training stages
# Train on progressively longer time windows so the network learns
# early-time dynamics first, then extends to the full domain.
curriculum_stages = [
    {"t_end": 1.0, "epochs": 5000},
    {"t_end": 2.0, "epochs": 5000},
    {"t_end": 3.0, "epochs": 5000},
    {"t_end": 4.0, "epochs": 5000},
    {"t_end": 5.0, "epochs": 5000},
]

# -------------------------------------------------------------------
# Instantiate the PINN model and set up training components
pinn = FullyConnectedNN(
    n_input, n_output, n_hidden, n_layers).to(device)

# Initial condition: t=0 -> theta1=45 deg, theta2=0 deg
t_ic = torch.tensor(
    [[0.0]], device=device, requires_grad=True)
N_phys  = 1000

optimiser = torch.optim.Adam(
    pinn.parameters(),
    lr=learning_rate)


scheduler = torch.optim.lr_scheduler.StepLR(
    optimiser, step_size=5000, gamma=0.5)


total_epochs = sum(s["epochs"] for s in curriculum_stages)
n_params = sum(p.numel() for p in pinn.parameters())
stages_summary = [
    (s['t_end'], s['epochs']) for s in curriculum_stages]

print(f"Network: {n_input} input -> "
      f"{n_layers}x{n_hidden} Tanh -> {n_output} outputs")
print(f"Parameters: {n_params}")
print(f"lambda_physics = {lambda_physics}  "
      f"(reduced to let data guide early training)")
print(f"Curriculum stages: {stages_summary}")
print(f"Total training epochs: {total_epochs}")

Network: 1 input -> 4x64 Tanh -> 2 outputs
Parameters: 12738
lambda_physics = 0.1  (reduced to let data guide early training)
Curriculum stages: [(1.0, 5000), (2.0, 5000), (3.0, 5000), (4.0, 5000), (5.0, 5000)]
Total training epochs: 25000

Part 1: Initial Condition (IC) Loss

1. Normalise and forward-pass through the network to get .
2. Denormalise both outputs to physical units.
3. Compute at via torch.autograd.grad. Because the network has two outputs, torch.ones_like(thetas_ic) sums the gradients across both columns (each is a scalar at the single IC point), giving a (1, 2) Jacobian row.
4. Penalise deviations from all four known initial values:

Part 2: Physics (Coupled ODE Residual) Loss

1. Normalise the 1,000 collocation times (within the current stage’s window) and forward-pass to get normalised predictions.
2. Denormalise to physical .
3. Compute first and second derivatives for each angle separately via four calls to torch.autograd.grad (one per derivative per angle). All use create_graph=True so the computational graph extends through the derivatives for backpropagation.
4. Evaluate both ODE residuals in physical space:

where .

Both residuals are included so the network is forced to satisfy both coupled ODEs simultaneously.

Part 3: Data Loss (in normalised space)

Pass the pre-normalised experimental times within the current stage’s window through the network. The MSE is computed across both angles:

Because each angle was normalised with its own statistics, the two columns contribute equally to the loss. As the curriculum window grows, more experimental data points are included.

Part 4: Total Loss and Weight Update

total_loss.backward() computes gradients, optimiser.step() updates weights via Adam, and scheduler.step() halves the learning rate every 5,000 epochs (coinciding with curriculum stage boundaries). All four loss components (total, IC, physics, data) are recorded at every epoch for the training loss plot.

total_loss_history = []
ic_loss_history = []
physics_loss_history = []
data_loss_history = []

start_training_time = time.time()

for stage_idx, stage in enumerate(curriculum_stages):
    t_end_stage   = stage["t_end"]
    stage_epochs  = stage["epochs"]

    # Build collocation points for the current time window
    t_phys = torch.linspace(
        t_start, t_end_stage, N_phys, device=device).view(-1, 1)
    t_phys.requires_grad_(True)

    # Select only experimental data within the current window
    mask       = t_exp_np <= t_end_stage
    t_exp_stage = t_exp_norm[mask]
    theta_exp_stage = theta_exp_norm[mask]

    print(f"Stage {stage_idx+1}/{len(curriculum_stages)} | "
          f"t ∈ [0, {t_end_stage}] s | {stage_epochs} epochs")

    for i in range(stage_epochs):
        
        optimiser.zero_grad()

        # --------------------------------------------------------------------
        # Part 1: Initial Condition Loss
        t_ic_norm      = normalise_t(t_ic)
        thetas_ic_norm = pinn(t_ic_norm)
        thetas_ic      = denormalise_theta(thetas_ic_norm)

        dthetas_dt_ic  = torch.autograd.grad(
            outputs=thetas_ic,
            inputs=t_ic,
            grad_outputs=torch.ones_like(thetas_ic),
            create_graph=True)[0]

        loss_theta_ic  = (
            torch.mean((thetas_ic[:, 0] - (45*torch.pi/180))**2)
            + torch.mean((thetas_ic[:, 1] - 0.0)**2))

        loss_dtheta_dt_ic = torch.mean(dthetas_dt_ic**2)
        total_loss_ic        = loss_theta_ic + loss_dtheta_dt_ic

        # --------------------------------------------------------------------
        # Part 2: Physics (Coupled ODE Residual) Loss
        t_phys_norm = normalise_t(t_phys)
        thetas_norm    = pinn(t_phys_norm)
        thetas         = denormalise_theta(thetas_norm)

        th1 = thetas[:, 0:1]
        th2 = thetas[:, 1:2]

        dth1_dt  = torch.autograd.grad(
            outputs=th1,
            inputs=t_phys,
            grad_outputs=torch.ones_like(th1),
            create_graph=True)[0]

        dth2_dt  = torch.autograd.grad(
            outputs=th2,
            inputs=t_phys,
            grad_outputs=torch.ones_like(th2),
            create_graph=True)[0]

        d2th1_dt2 = torch.autograd.grad(
            outputs=dth1_dt,
            inputs=t_phys,
            grad_outputs=torch.ones_like(dth1_dt),
            create_graph=True)[0]
        
        d2th2_dt2 = torch.autograd.grad(
            outputs=dth2_dt,
            inputs=t_phys,
            grad_outputs=torch.ones_like(dth2_dt),
            create_graph=True)[0]

        delta = th2 - th1

        # Compute the residuals of the ODEs (r1 and r2 should ideally be zero)
        r1 = (
            (m1+m2)*L1*d2th1_dt2
            + m2*L2*d2th2_dt2*torch.cos(delta)
            - m2*L2*dth2_dt**2*torch.sin(delta)
            + (m1+m2)*g*torch.sin(th1))

        r2 = (
            L2*d2th2_dt2
            + L1*d2th1_dt2*torch.cos(delta)
            + L1*dth1_dt**2*torch.sin(delta)
            + g*torch.sin(th2))

        loss_physics = torch.mean(r1**2) + torch.mean(r2**2)

        # --------------------------------------------------------------------
        # Part 3: Data Loss (normalised space)
        thetas_data_norm = pinn(t_exp_stage)
        loss_data        = torch.mean(
            (thetas_data_norm - theta_exp_stage)**2)

        # --------------------------------------------------------------------
        # Part 4: Total Loss
        total_loss = (
            lambda_ic * total_loss_ic
            + lambda_physics * loss_physics
            + lambda_data * loss_data)

        total_loss.backward()
        optimiser.step()
        scheduler.step()

        total_loss_history.append(total_loss.item())
        ic_loss_history.append(total_loss_ic.item())
        physics_loss_history.append(loss_physics.item())
        data_loss_history.append(loss_data.item())

        if i % 1000 == 0 or i == stage_epochs - 1:
            print(f"  Epoch {i:>5d}/{stage_epochs}  " 
                  f"Loss: {total_loss.item():.6f}")

end_training_time = time.time()
training_duration = end_training_time - start_training_time

print(f"\nTraining complete. Final loss: {total_loss_history[-1]:.6f}")
print(f"Total training time: {training_duration:.2f} seconds")

Stage 1/5 | t ∈ [0, 1.0] s | 5000 epochs
  Epoch     0/5000  Loss: 1.937241
  Epoch  1000/5000  Loss: 0.000584
  Epoch  2000/5000  Loss: 0.000591
  Epoch  3000/5000  Loss: 0.000609
  Epoch  4000/5000  Loss: 0.000634
  Epoch  4999/5000  Loss: 0.000471
Stage 2/5 | t ∈ [0, 2.0] s | 5000 epochs
  Epoch     0/5000  Loss: 3.019290
  Epoch  1000/5000  Loss: 0.000696
  Epoch  2000/5000  Loss: 0.000476
  Epoch  3000/5000  Loss: 0.000428
  Epoch  4000/5000  Loss: 0.000890
  Epoch  4999/5000  Loss: 0.000414
Stage 3/5 | t ∈ [0, 3.0] s | 5000 epochs
  Epoch     0/5000  Loss: 2.086826
  Epoch  1000/5000  Loss: 0.000753
  Epoch  2000/5000  Loss: 0.000500
  Epoch  3000/5000  Loss: 0.000434
  Epoch  4000/5000  Loss: 0.000397
  Epoch  4999/5000  Loss: 0.000375
Stage 4/5 | t ∈ [0, 4.0] s | 5000 epochs
  Epoch     0/5000  Loss: 2.054655
  Epoch  1000/5000  Loss: 0.033731
  Epoch  2000/5000  Loss: 0.001052
  Epoch  3000/5000  Loss: 0.000682
  Epoch  4000/5000  Loss: 0.000568
  Epoch  4999/5000  Loss: 0.000470
Stage 5/5 | t ∈ [0, 5.0] s | 5000 epochs
  Epoch     0/5000  Loss: 1.595923
  Epoch  1000/5000  Loss: 0.081953
  Epoch  2000/5000  Loss: 0.015572
  Epoch  3000/5000  Loss: 0.001536
  Epoch  4000/5000  Loss: 0.001006
  Epoch  4999/5000  Loss: 0.000679

Training complete. Final loss: 0.000679
Total training time: 316.82 seconds

PINN displacement prediction

500 evenly spaced test points are generated over s, normalised, and passed through the trained network inside torch.no_grad() (no gradient tracking needed at inference). The two-column normalised output is denormalised back to physical angles for plotting.

Plot 1: Displacement comparison (stacked)

Two vertically stacked subplots, one per pendulum arm, each overlaying three elements:

• PINN prediction (black solid line): the trained network’s angle .
• Numerical solution (dashed line): the ground-truth odeint result. If training succeeded, the two curves should be nearly indistinguishable.
• Experimental data (coloured scatter): the 250 noisy observations used for training.

Plot 2: Training loss breakdown

Four loss curves on a log scale over 25,000 epochs (five curriculum stages of 5,000 epochs each):

• Total loss (black): the weighted sum .
• IC loss (orange): deviation from the four initial conditions.
• Physics loss (red): mean squared ODE residual across collocation points.
• Data loss (blue): MSE between predicted and observed normalised angles.

Orange dashed vertical lines mark the LR reduction points (every 5,000 epochs, coinciding with curriculum stage boundaries). Each reduction should produce a visible change in convergence behaviour as the optimiser enters a finer learning regime.

# Evaluate PINN
t_test      = torch.linspace(t_start, t_end, 500, device=device).view(-1, 1)
t_test_norm = normalise_t(t_test)

with torch.no_grad():
    thetas_pred_norm = pinn(t_test_norm)
    thetas_pred      = denormalise_theta(thetas_pred_norm)

t_plot   = t_test[:, 0].detach().cpu().numpy()
th1_pinn = thetas_pred[:, 0].detach().cpu().numpy()
th2_pinn = thetas_pred[:, 1].detach().cpu().numpy()

# ----------------------------------------------------------------------------
# Plot 1: Displacement — both arms
fig, axes = plt.subplots(
    2, 1,
    figsize=(9, 10))

axes[0].plot(
    t_plot,
    th1_pinn,
    color='black',
    linewidth=2,
    label=r'PINN $\hat\theta_1$')

axes[0].plot(
    t,
    theta1,
    color='blue',
    linestyle='--',
    alpha=0.7,
    label=r'Numerical $\theta_1$')

axes[0].scatter(
    t_exp_np,
    theta1_exp_np,
    s=15,
    color='blue',
    alpha=0.8,
    label='Exp Data')

axes[0].set_xlabel(
    'Time (s)',
    fontsize=12)
axes[0].set_ylabel(
    'Angle (rad)',
    fontsize=12)
axes[0].set_title(
    r'Top Pendulum Dynamics ($\theta_1$)',
    fontsize=13)
axes[0].legend(fontsize=10)
axes[0].grid(True, alpha=0.3)

axes[1].plot(
    t_plot,
    th2_pinn,
    color='black',
    linewidth=2,
    label=r'PINN $\hat\theta_2$')
axes[1].plot(
    t,
    theta2,
    color='red',
    linestyle='--',
    alpha=0.7,
    label=r'Numerical $\theta_2$')
axes[1].scatter(
    t_exp_np,
    theta2_exp_np,
    s=15,
    color='red',
    alpha=0.8,
    label='Exp Data')
axes[1].set_xlabel(
    'Time (s)', fontsize=12)
axes[1].set_ylabel('Angle (rad)', fontsize=12)
axes[1].set_title(r'Bottom Pendulum Dynamics ($\theta_2$)', fontsize=13)
axes[1].legend(fontsize=10); axes[1].grid(True, alpha=0.3)

plt.tight_layout()
plt.show()

# ----------------------------------------------------------------------------
# Plot 2: Training loss
fig, ax = plt.subplots(figsize=(9, 5))

ax.plot(
    total_loss_history,
    marker='o',
    markersize=2,
    color='black',
    linewidth=3,
    label="Total Loss")
ax.plot(
    ic_loss_history,
    color='orange',
    linewidth=0.8,
    label="IC Loss")
ax.plot(
    physics_loss_history,
    color='red',
    linewidth=1.2,
    alpha=0.8,
    label="Physics Loss")
ax.plot(
    data_loss_history,
    color='blue',
    linewidth=1.2,
    alpha=0.8,
    label="Data Loss")


for ep in [5000, 10000, 15000]:
    plt.axvline(
        x=ep,
        color='orange',
        linestyle='--',
        alpha=0.5)

ax.axvline(
    x=5000,
    color='orange',
    linestyle='--',
    alpha=0.5,
    label='LR reductions')

ax.set_yscale('log')
ax.set_xlabel('Epoch', fontsize=12)
ax.set_ylabel('Total Loss (log scale)', fontsize=12)
ax.set_title('2-DOF Hybrid PINN Training Loss', fontsize=13)
ax.legend(fontsize=11)
ax.grid(True, which='both', linestyle='-', alpha=0.2)

plt.tight_layout()
plt.show()

Term 1: Initial Condition (IC)/Boundary Condition (BC) Loss

penalises the network for predicting the wrong starting angle or starting angular velocity at . It is purely a boundary/initial constraint.

Constant parameter in the ODE-model

With a fixed pendulum length, the network only takes as input, so there is a single IC to satisfy:

Flexible parameter in the ODE-model

Because the network now takes two inputs , the starting conditions must hold for all values in the training range. A single BC check at one would leave the network free to predict wrong initial angles at other lengths. The sum over sampled values of ensures the network learns to start at with velocity regardless of which pendulum length is queried:

Term 2: Physics (Residual) Loss

Constant parameter in the ODE-model

With a fixed , the coefficient is a constant computed once before training. The residual only depends on :

where is the physics loss that penalises the network for violating the equation of motion at the collocation points.

Flexible parameter in the ODE-model

When varies, can no longer be precomputed as a single constant. Instead, each collocation point carries its own , and the ratio must be evaluated per point inside the residual. The network predictions , , and now all depend on both and :

This ensures the ODE is enforced across the full domain, not just for one pendulum length.

Total Loss

Model architecture

• Two-element input: at each forward pass, the network receives a pair , the time value and the pendulum length at that collocation point, concatenated into a 2-element input vector. During evaluation, all time points share the same fixed value (the chosen test length).

• Tanh activation is still required: we still need to compute via automatic differentiation with respect to , which requires the network to be twice differentiable in . Tanh is infinitely smooth, so this is satisfied. We do not differentiate with respect to (it is a parameter, not a variable to differentiate through), but smooth derivatives in remain essential.

• Minimal capacity change: adding one input neuron to the first layer increases the parameter count by only 64 weights (one extra weight per hidden neuron in the first layer) out of roughly 8,000+ total parameters. This is less than a 1% increase. The network’s representational capacity is determined by the width (64) and depth (3 layers), not by whether the input is 1D or 2D.

Define the Model Parameters

t_start = 0.0
t_end   = 1.0

g  = 9.81          # gravitational acceleration (m/s^2)
L  = 1.0           # pendulum length (m)
m  = 1.0           # mass (kg)
b  = 0.01          # damping coefficient (kg/s)

# Initial angle (radians)
theta0 = 1.0
  
# Initial angular velocity
# starts from rest, so angular velocity is zero
omega0 = 0.0

# L is now a range, not a single value
# minimum pendulum length (metres)
L_min  = 0.5     

# maximum pendulum length (metres)
L_max  = 1.5     

Network architecture

We use a fully connected (dense) feedforward neural network with tanh activations. The architecture has three parts:

Layer block	Abbreviation	Description
fcs	fully connected start	Input layer: maps inputs to N_NEURON neurons, followed by tanh.
fch	fully connected hidden	Hidden layers: N_LAYERS - 1 identical blocks, each a linear layer + tanh.
fce	fully connected end	Output layer: a single linear layer that maps from N_NEURON to the single output .

Here we use N_NEURON = 64 and N_LAYERS = 3, giving two hidden blocks and one input block (three tanh layers total).

Each nn.Linear(in, out) layer computes , where is an out in weight matrix and is a bias vector of length out. For example, nn.Linear(1, 64) takes the single scalar input and produces a 64-dimensional vector, one value per hidden neuron.

Number of layers

N_LAYERS represents the total number of layers with activations. The fcs block already provides the first activated layer, so fch only needs to create the remaining N_LAYERS - 1. With N_LAYERS = 3:

class FullyConnectedNN(nn.Module):
    def __init__(self, N_INPUT, N_OUTPUT, N_NEURON, N_LAYERS):
        super().__init__()

        # Using Tanh activation
        activation = nn.Tanh

        # First layer: input to hidden layers
        # Using nn.Sequential to stack layers and activations
        self.fcs = nn.Sequential(
            nn.Linear(N_INPUT, N_NEURON), activation())

        # Hidden layers: N_LAYERS - 1 hidden layers
        # Since fcs already provides one, fch only needs to add the 
        # remaining N_LAYERS - 1.
        self.fch = nn.Sequential(*[
            nn.Sequential(nn.Linear(N_NEURON, N_NEURON), activation())
            for _ in range(N_LAYERS - 1)])

        # Output layer: hidden to output
        self.fce = nn.Linear(N_NEURON, N_OUTPUT)

    # Forward pass through the network
    def forward(self, x):
        x = self.fcs(x)
        x = self.fch(x)
        x = self.fce(x)
        return x

Training Setup

The network is instantiated with N_INPUT = 2 (instead of 1) so the first linear layer accepts the concatenated vector . Everything else in the architecture stays the same.

Boundary condition points (IC loss)

We sample 500 random values from and pair each with . This ensures the initial conditions and ( are enforced across the full range of pendulum lengths, not just at a single .

NoteUniform sampling from

The notation is called a half-open interval. The square bracket means the left endpoint is included, and the round parenthesis means the right endpoint is excluded. So contains every number satisfying : zero is a possible value, but one is not. This is exactly the range produced by torch.rand.

Both L_phys and L_bc are generated using the pattern:

L_phys = torch.rand(N, 1) * (L_max - L_min) + L_min
L_bc = torch.rand(N, 1) * (L_max - L_min) + L_min

This samples uniformly from in three steps:

1. torch.rand(N, 1) produces random values in .
2. Multiplying by scales the range to .
3. Adding shifts it to .

In general: to sample uniformly from .

t_phys uses the same idea but with :

t_phys = torch.rand(N_phys, 1, device=device) * t_end

When the formula simplifies to . torch.rand gives values in , and multiplying by t_end scales them to . No shift is needed because the interval already starts at zero.

# Set random seed for reproducibility
torch.manual_seed(42)

# Note: n_input is now 2 to account for (t, L)
n_input   = 2

# n_output is still 1 since we're predicting theta(t, L)
n_output  = 1
n_hidden  = 64
n_layers  = 3
epochs    = 10000
learning_rate = 1e-3

# ----------------------------------------------------------------------------
# Create the PINN model
pinn = FullyConnectedNN(
    n_input,
    n_output,
    n_hidden,
    n_layers).to(device)

# Create the optimiser using Adam algorithm
optimiser = torch.optim.Adam(
    pinn.parameters(),
    lr=learning_rate)

# ----------------------------------------------------------------------------
# Collocation points: randomly sample (t, L) space
N_phys = 2000
# t_phys is sampled uniformly from [0, t_end)
t_phys = torch.rand(N_phys, 1, device=device) * t_end
t_phys.requires_grad_(True)

# L_phys is sampled uniformly from [L_min, L_max)
L_phys = torch.rand(
    N_phys, 1, device=device) * (L_max - L_min) + L_min

# ----------------------------------------------------------------------------
# Boundary points: t=0 for 500 random L values
N_boundary = 500

# L_bc is sampled uniformly from [L_min, L_max)
L_bc = torch.rand(
    N_boundary, 1, device=device) * (L_max - L_min) + L_min

# t_bc is set to 0 for all boundary points
t_bc = torch.zeros_like(
    L_bc, device=device, requires_grad=True)


print(f"Collocation points: {N_phys}   Boundary points: {N_boundary}")
print(f"Training for {epochs} epochs...")

Collocation points: 2000   Boundary points: 500
Training for 10000 epochs...

Inspecting PINN parameters in the network

pinn.parameters() is a generator that yields every learnable tensor (weight matrix or bias vector) in the network. For each parameter tensor p:

• p.shape shows its dimensions (e.g. torch.Size([64, 2]) for the first weight matrix, which maps 2 inputs to 64 neurons).
• p.numel() returns the total number of scalar values in that tensor (i.e. the product of all dimensions).

sum(p.numel() for p in pinn.parameters()) adds up the element counts across all tensors to give the total number of trainable parameters in the network.

With N_INPUT = 2, N_NEURON = 64, N_LAYERS = 3, and N_OUTPUT = 1, the loop prints eight tensors (one weight matrix and one bias vector per linear layer):

Block	Layer	Tensor	Shape	Parameters
fcs	Linear(2, 64)	weight		128
-	-	bias		64
fch[0]	Linear(64, 64)	weight		4,096
-	-	bias		64
fch[1]	Linear(64, 64)	weight		4,096
-	-	bias		64
fce	Linear(64, 1)	weight		64
-	-	bias		1
-	-	-	Total	8,577

Note that the fcs weight matrix has shape rather than because N_INPUT = 2. This is the only layer affected by the change from 1D to 2D input, adding just 64 extra weights compared to the single-input PINN (which has weights in the first layer, totalling 8,513 parameters).

for index, p in enumerate(pinn.parameters(), 1):
    print(f"Layer {index}: {p.numel()} parameters with shape: {p.shape}")

print("--" * 50)
print("Total parameters in the network:")
print(sum(p.numel() for p in pinn.parameters()))

Layer 1: 128 parameters with shape: torch.Size([64, 2])
Layer 2: 64 parameters with shape: torch.Size([64])
Layer 3: 4096 parameters with shape: torch.Size([64, 64])
Layer 4: 64 parameters with shape: torch.Size([64])
Layer 5: 4096 parameters with shape: torch.Size([64, 64])
Layer 6: 64 parameters with shape: torch.Size([64])
Layer 7: 64 parameters with shape: torch.Size([1, 64])
Layer 8: 1 parameters with shape: torch.Size([1])
----------------------------------------------------------------------------------------------------
Total parameters in the network:
8577

Training Loop

Each epoch performs three steps. The structure is the same as the fixed- PINN, but every forward pass now receives a 2D input instead of just , and is computed per collocation point rather than once before training.

total_loss_history = []
bc_loss_history = []
phys_loss_history = []

start_time = time.time()

for i in range(epochs + 1):
    optimiser.zero_grad()

    # ------------------------------------------------------------------------
    # Part 1: Boundary Condition (BC) Loss
    # concatenate t_bc and L_bc → shape (N_boundary, 2)
    # Each row is one (t=0, L_j) pair
    # the network sees both inputs simultaneously
    inputs_bc    = torch.cat([t_bc, L_bc], dim=1)
    theta_bc     = pinn(inputs_bc)
    dtheta_dt_bc = torch.autograd.grad(
        outputs=theta_bc,
        inputs=t_bc, 
        grad_outputs=torch.ones_like(theta_bc),
        create_graph=True)[0]

    loss_bc = torch.mean((theta_bc - theta0)**2) + torch.mean(
        (dtheta_dt_bc - omega0)**2)

    # ------------------------------------------------------------------------
    # Part 2: Physics (ODE Residual) Loss
    inputs_phys      = torch.cat([t_phys, L_phys], dim=1)
    theta_phys       = pinn(inputs_phys)
    dtheta_dt_phys   = torch.autograd.grad(
        outputs=theta_phys,
        inputs=t_phys,
        grad_outputs=torch.ones_like(theta_phys),
        create_graph=True)[0]
    d2theta_dt2_phys = torch.autograd.grad(
        outputs=dtheta_dt_phys,
        inputs=t_phys,
        grad_outputs=torch.ones_like(dtheta_dt_phys),
        create_graph=True)[0]

    # Using a different value at each collocation point
    # cp = g / L_phys 
    # This correctly encodes the physics for each individual pendulum length
    cp  = g / L_phys

    # ODE residual = d²θ/dt² + (b/m)·dθ/dt + cp·sin(θ)
    # cp is now a tensor (N, 1), so this is an element-wise operation
    res = d2theta_dt2_phys + (b/m)*dtheta_dt_phys + cp*torch.sin(theta_phys)

    loss_phys = torch.mean(res**2)

    # ------------------------------------------------------------------------
    # Part 3: Total Loss
    total_loss = loss_bc + loss_phys
    total_loss.backward()
    optimiser.step()

    # Store the loss values for plotting later
    total_loss_history.append(total_loss.item())
    bc_loss_history.append(loss_bc.item())
    phys_loss_history.append(loss_phys.item())

elapsed_time_pinn_training = time.time() - start_time
print(f"\nTraining complete in {elapsed_time_pinn_training:.4f} seconds.")
print(f"Final loss: {total_loss_history[-1]:.6f}")

Training complete in 91.9363 seconds.
Final loss: 0.000513

Why Are There Spikes in the Loss?

The training loss plot above shows dramatic, regularly-spaced spikes that persist throughout training. There are several contributing factors:

1. Gradient competition between loss terms (primary cause)

The two losses (i.e., BC loss and physics loss) pull the network in opposing directions. When Adam makes good progress minimising the physics residual, it can push the weights into a region where the boundary conditions are violated (and vice versa). With equal weighting ( for both) and no adaptive balancing, these competing gradients periodically destabilise each other, producing the sawtooth pattern.

2. Fixed collocation points (no stochastic smoothing)

The 2,000 collocation points and 500 boundary points are sampled once before training and reused every epoch. There is no resampling. This means the loss landscape is deterministic and the optimiser can get trapped oscillating in narrow valleys. Periodic resampling of collocation points would introduce stochastic averaging that dampens these oscillations.

3. Constant learning rate with Adam’s momentum

Adam at a fixed lr = 1e-3 accumulates momentum. When progress on one loss term builds up momentum, the resulting step can overshoot and spike the other loss. Without learning rate decay or gradient clipping, there is nothing to dampen these overshoots as training progresses.

Possible mitigations:

• Resample collocation points every epochs to smooth the loss landscape.
• Learning rate scheduling (e.g. cosine decay or ReduceLROnPlateau) to reduce step sizes as training progresses.
• Adaptive loss weighting (e.g. the method of Wang et al., 2021) so the two loss terms remain balanced.
• Gradient clipping (torch.nn.utils.clip_grad_norm_) to prevent large update steps.